Autonomous highway driving is a very different animal than autonomous driving in a city. The highway driving is relatively easy when compared to city driving. There's a bunch of other driving situations that have very different levels of complexity.

I just think about how many accidents that I have avoided in the past. There have been times when I just knew that someone at the stop sign was going to blow thru it, I couldn't tell you how I knew but I did. I have seen other instances where I just knew someone was going to do something total boneheaded, again, there were no obvious clues but my brain picked up on them. As humans we process a ton of data in real time, when there isn't enough data, our brains fill in the blanks. We have intuition, I have no clue how you would make any machine have intuition. I just don't know how you could ever program for those kinds of things.

In the end, I don't think that we will ever have a fully autonomous driving system that works in all situations. Yeah, I know, humans do make a lot of mistakes and maybe autonomous cars will make fewer mistakes, I don't know.

A bigger concern is how my autonomous car would make decisions. What are the priorities? Is the life of the passengers top priority? Will my car sacrifice me to save other persons or people? The questions can get into some pretty philosophical/moral areas.