Because this sort of auto->manual transition of control does not work [1].

1: https://www.amazon.com/Our-Robots-Ourselves-Robotics-Autonom...

The problem you are stating is called the 'handoff' problem which I believe will be a serious challenge for manufacturers of driverless cars. The handoff problem is when an automated system identifies a scenario in which it needs to hand over control to a human. How will that work in a vehicle that is not designed to be occupied by someone that knows how to control it? This problem has been written about in the book Our Robots, Ourselves [0] and there is an Econ Talk podcast about it as well [1].

To me, it makes sense to make use of the brain power provided by the occupant of the vehicle. For example, if one of the main vision sensors in the car breaks, why not let the human occupant drive the car manually?

[0] http://www.amazon.co.uk/Our-Robots-Ourselves-Robotics-Autono...

[1] http://www.econtalk.org/archives/2015/11/david_mindell_o.htm...