Automotive vision looks to put technology in the driver’s seat

One could argue that there is a fourth level—that of monitoring the driver. This concept is used in commercial vehicles to monitor the activities of the driver and or vehicle while it is in use. But we will not consider this further here.

Occupant monitoring. If asked to define the particulars of automotive vision, the average person would not likely suggest that the car should observe its occupants. But this could easily be one for the first automotive vision applications. It is expected to yield impressive safety results; and since the car’s interior is a predictable environment, it is also a relatively straightforward implementation with the possible exception of the inference software required.

Monitoring drivers for drowsiness, inattentiveness, or intoxication is likely to roll out in commercial fleet vehicles as early as 2006. A more difficult aspect of occupant monitoring calls for the vehicle to observe the position and posture of the passengers and, in the case of an accident, deploy airbags to account accordingly. Still in the research phase, vision-based smart airbag deployment will take several years to perfect.

Infrastructure monitoring. A vehicle’s ability to recognize and interact with stationary objects that exist around it or have been added as part of the highway infrastructure can add to the safety of the occupants of the vehicle and the convenience of the driver. Such things as embedded RFID chips in the ubiquitous reflectors set in traffic lanes, for example, could enable the car to alert the driver that it has drifted out of its lane. Or, with that RFID or vision-based information, the car could actually steer itself back into the proper lane. Similarly, sensors such as cameras or radar units mounted on the car could detect objects such as median barriers, trees, buildings, and people, in addition to those equipped with sensors.

Other specific applications that fall into the category of infrastructure monitoring include: parking assist proximity sensors for parallel parking (already in production in Europe); rear view cameras instead of mirrors; and blind spot cameras, which are also useful in monitoring other vehicles. Still another aspect of infrastructure monitoring is the interaction of automotive vision technology with existing technologies such as GPS. Knowing a vehicle’s position could be helpful in accident avoidance if, for example, it could advise of an accident or road hazard in the vicinity.

Monitoring other vehicles. This category is distinguished from the previous category by the inherent difficulty of tracking other moving objects. Examples include intelligent cruise control (to maintain a safe distance between vehicles), blind spot monitoring, rear view observation, and night vision.

Safety and forensics. Some applications do not fit easily fit into the three categories mentioned above. Black-box recorders can be used for accident reconstruction, for example, are already being used in some commercial applications. Technologies that will help autonomous vehicles learn how to avoid an accident or how to react in a crash situation are being developed in research labs.

Data fusion
The term automotive vision suggests mimicking the visual perception of the human eye with cameras. But vehicles encounter many conditions in which a “few extra eyeballs” distributed around the vehicle are not enough to provide all the safety and convenience possible.

There is general agreement among engineers in the automotive and associated industries that a variety of sensors can be used to collect a more complete set of useful information. These sensors could possibly include radar, laser, and infrared in addition to digital cameras. Data from all of these sensors will be combined and interpreted in a concept called data fusion.

In the near term, however, the focus will be on cameras because they offer good price/performance and can effectively collect-and-display information for the driver. Data fusion will be an important aspect of automotive vision in the future; but for this article the types of data are not as important as how the data are used. The industry is still grappling with several questions about the location of sensors, their resolution, and where the intelligence that will interpret the data collected should reside.