How to implement automotive smart rear-view cameras

The National Highway Transportation Safety Authority (NHTSA) estimates that back-up accidents are the cause of 292 deaths and approximately 18,000 injuries per year. The Cameron Gulbransen Kids Transportation Safety Act of 2007 – named for a 2-year old child tragically killed when his father accidentally backed over him in the family’s driveway –  has spurred auto safety regulators to set standards for rear visibility for passenger vehicles, which have never previously been regulated. NHTSA is mandating rear-view cameras be installed in all light trucks and cars by 2014. 

Legislation for mandatory rear-view cameras is still pending, yet vehicles have become larger and the ability to see people or objects behind them has been reduced. Other ways of improving rear visibility have been studied, including radar based sensors, however it was determined that these sensors often did not detect moving people, especially children. Studies show that drivers responded better to a camera image than audio alerts and market research indicates that a subset of back-up cameras will incorporate embedded vision to automatically detect objects, estimate their distance to the vehicle’s bumper, and warn drivers visually and audibly of an impending collision.

This article describes an embedded vision system for smart back-up camera with object detection, tracking and distance estimation.  The requirements, challenges and outcome of creating a smart rear-view camera system designed to reduce back-overs, fatalities and injuries are presented.

An Introduction to Embedded Vision
Embedded vision is the merging of two technologies: embedded systems and computer vision. An embedded system is a microprocessor-based system that isn’t a general-purpose computer, and is designed to handle a particular task. Embedded systems are ubiquitous – they’re found in automobiles, appliances, consumer electronics, medical equipment, and in many other aspects of our daily lives. Computer vision (also referred to as “machine vision”) is a field that uses digital processing and algorithms to electronically perceive and interpret meaning from images or video. While computer vision has mainly been a field of academic research over the past several decades, today a major transformation is underway. With the emergence of programmable high performance, low cost, low power processors capable of running algorithms to extract relevant application-specific information,  it is now possible to incorporate vision capabilities into a wide range of embedded systems.

Embedded vision development requires a multi-phased approach. The following figure illustrates a typical embedded vision application development flow which was used in the development of the smart back-up camera application.




Like in most development flows, test and validation of the algorithm at various stages is essential to ensure expected response. As such, before all else, a test and validation video database must be acquired. For many vision applications, there are standard databases available, however oftentimes a relevant and comprehensive suite of testcases does not exist. If this is the case (as it was for our smart back-up camera system), video clips that make up the validation test suite must be captured one by one. This proved to be a huge undertaking.

Video sequences in all weather conditions, and lighting conditions were captured.  As the validation database was built-up, it was critical to setup an automated validation procedure to run unsupervised and report on the statistical performance of the algorithm.