The Bosch system may go into production in 2022. In that year, the EU will make safety technology that warns drivers of drowsiness and distraction, for example, a standard feature in new vehicles. The EU Commission expects the new safety requirements for vehicles will save more than 25,000 lives and help prevent at least 140,000 severe injuries by 2038.
By tracking what is happening inside the car, technology can help solve a fundamental problem of self-driving cars. For example, if responsibility for driving is to be transferred to the driver following an automated drive, the car needs to be able to prove the driver is neither sleeping, nor reading the newspaper, nor writing emails on their smartphone.
A smart camera constantly monitors the driver
If a driver dozes off or looks at a smartphone for just three seconds, a vehicle traveling at approximately 30mph can cover about 130 feet completely unsupervised, and many people underestimate the associated risk. International studies state that nearly one in 10 accidents are caused by distraction or drowsiness, which has prompted Bosch to develop an interior monitoring system that detects and alerts to this danger and provides driving assistance. A camera integrated in the steering wheel detects when drivers’ eyelids are getting heavy, when they are distracted, and when they turn their head toward passenger or rear seats. Thanks to AI, the system draws the right conclusions from this information: it warns inattentive drivers, recommends a break if they are getting tired, or even reduces the speed of the vehicles – depending on the automaker’s wishes, as well as legal requirements.
“Cameras and AI will turn the vehicle into a lifesaver,” Kroeger says. To achieve this, Bosch engineers have used intelligent image-processing algorithms and machine learning to teach the system to understand what the person in the driving seat is doing. For driver drowsiness, the system is trained using recordings of real driving situations, eyelid position and eye-blink rate to learn how tired the driver really is. This allows the system to give an alert that is appropriate to the situation, so driver assistance can be used to intervene.
Warning systems that sound the alert, in the case of distraction and drowsiness, will be so important in the future that NCAP, the European New Car Assessment Program, will include them in the roadmap for its vehicle safety assessment by 2025. Further, only the software in the vehicle itself evaluates the information provided by the interior monitoring system – the information is neither saved nor shared with third parties.
Like a relay race, responsibility for steering passes from car to driver and back
At the very least when cars start driving automatically, it is important that they understand their drivers, as cars will drive along freeways without driver intervention. They will also have to hand back control to their drivers in tricky situations such as construction zones or when the exit ramp is drawing near. Drivers need the option to safely take the wheel again at any time during the automated driving phase. The camera makes sure they don’t fall asleep, and if their eyes remain closed for a prolonged period, an alarm is sounded. The system also interprets camera recordings to establish what drivers are currently doing and how ready they are to respond. The transfer of driving responsibility is then timed accordingly. “Bosch driver observation will be essential for safe automated driving,” Kroeger says.
When the car keeps its camera eyes open
The new Bosch system keeps its eye not only on the driver, it also tracks all the other passengers, whether next to or behind the driver. For this purpose, a camera mounted above or below the rearview mirror monitors the entire passenger compartment. It notices when children in the rear seats have carelessly unfastened their seat belts and warns the driver. If someone sitting in the back is leaning too far forward, at an angle, or with their feet up on the seat next to them, the airbags and belt tensioner will not be able to protect them properly in an accident. The interior monitoring camera can tell what position they are sitting in and set the airbags and belt tensioner to ensure the best possible protection.
On the subject of children, it is a sad fact that parked vehicles can be dangerous for them. In the United States in 2018, parked vehicles claimed the lives of more than 50 children (source: KidsAndCars.org), either because they had been left in the car for a short while or had clambered in unnoticed. The new Bosch system can recognize this danger and warn parents quickly by sending a message to their smartphone. In an emergency, it also can alert the emergency services. As the Hot Cars Act currently being debated in the United States shows, legislators are interested in technology solutions to address this challenge.
A camera for more convenience
The Bosch interior monitoring camera can recognize preset driver preferences, such as seat position, rearview mirror adjustment, steering wheel height and infotainment profiles, for increased driver convenience. The camera can be used for eye- and hand-gesture control of the infotainment system as well.
