News & Analysis
Comment
Alix
I believe that vision in intelligent systems must be properly (intelligently?) ...
selinz
Safety, convenience, time saving. Oh, and play, I suppose (for example, all the ...
The next killer app: Machines that see
Jeff Bier
3/30/2012 11:37 AM EDT
Do embedded processors shape applications, or is it the other way around?
In reality, it works both ways. This is particularly evident in digital-signal-processing-intensive applications, such as wireless communications and video compression. These applications became feasible on a large scale only after the emergence of processors with adequate performance and sufficiently low prices and power consumption. And once those processors emerged, these applications started to take off. Then, the growing market attracted competition and investment. Processor vendors tuned their processors for these applications.
As a result, we got a generation of DSP processors with features such as add-compare-selection instructions for Viterbi decoding, and a generation of DSPs and CPUs with features like sum-of-absolute-difference instructions and single-instruction-multiple-data operations for video compression.
A few years ago, after nearly two decades of evaluating and using embedded processors for digital-signal-processing-intensive applications, my colleagues and I at BDTI realized that embedded computer vision applications were poised to benefit from the same type of “virtuous circle” that had previously enabled the proliferation of wireless communications and video compression algorithms.
Computer vision has been around for decades in applications like factory automation. But only very recently has vision begun to be incorporated into high-volume applications like video games and automobile safety systems. And, now that vision is starting to appear in volume applications, processor vendors are beginning to focus on embedded vision applications, and to tune their processors for these applications – often by incorporating specialized coprocessors specifically designed for vision processing.
It’s easy to see why processor suppliers are excited about embedded vision applications. “Machines that see” offer compelling value in many applications and markets. Take automotive safety, for example. Over one million people are killed each year in automobile accidents. By reducing the number and severity of collisions, vision-based safety systems may be able to save many thousands of lives.
Embedded vision also promises to improve human-machine interaction—long the Achilles' heel of consumer electronics. Instead of hunting for the right hand-held remote control, imagine a world where you simply stare at your TV for a few seconds, and in response it turns itself on and offers you a personalized menu of options, which you can choose from via simple gestures. Market research firm IMS Research estimates that by 2015, vision-enabled devices will be shipping at a rate of over 3 billion units per year.(Read about many more embedded vision applications here).
In some applications, vision functions will be relatively simple and will be able to fit into existing processors (perhaps with a modest boost in clock rate or an additional core). But many of the most compelling embedded vision applications use very performance-hungry algorithms. Implementing these algorithms at low cost and low power consumption will require specialized processors. As a result, we expect to see processor suppliers introducing more processors that are optimized for vision applications, and providing more application development support (such as optimized software libraries) for these applications.
Jeff Bier is founder, Embedded Vision Alliance and president, BDTI.
In reality, it works both ways. This is particularly evident in digital-signal-processing-intensive applications, such as wireless communications and video compression. These applications became feasible on a large scale only after the emergence of processors with adequate performance and sufficiently low prices and power consumption. And once those processors emerged, these applications started to take off. Then, the growing market attracted competition and investment. Processor vendors tuned their processors for these applications.
As a result, we got a generation of DSP processors with features such as add-compare-selection instructions for Viterbi decoding, and a generation of DSPs and CPUs with features like sum-of-absolute-difference instructions and single-instruction-multiple-data operations for video compression.
A few years ago, after nearly two decades of evaluating and using embedded processors for digital-signal-processing-intensive applications, my colleagues and I at BDTI realized that embedded computer vision applications were poised to benefit from the same type of “virtuous circle” that had previously enabled the proliferation of wireless communications and video compression algorithms.
Computer vision has been around for decades in applications like factory automation. But only very recently has vision begun to be incorporated into high-volume applications like video games and automobile safety systems. And, now that vision is starting to appear in volume applications, processor vendors are beginning to focus on embedded vision applications, and to tune their processors for these applications – often by incorporating specialized coprocessors specifically designed for vision processing.
It’s easy to see why processor suppliers are excited about embedded vision applications. “Machines that see” offer compelling value in many applications and markets. Take automotive safety, for example. Over one million people are killed each year in automobile accidents. By reducing the number and severity of collisions, vision-based safety systems may be able to save many thousands of lives.
Embedded vision also promises to improve human-machine interaction—long the Achilles' heel of consumer electronics. Instead of hunting for the right hand-held remote control, imagine a world where you simply stare at your TV for a few seconds, and in response it turns itself on and offers you a personalized menu of options, which you can choose from via simple gestures. Market research firm IMS Research estimates that by 2015, vision-enabled devices will be shipping at a rate of over 3 billion units per year.(Read about many more embedded vision applications here).
In some applications, vision functions will be relatively simple and will be able to fit into existing processors (perhaps with a modest boost in clock rate or an additional core). But many of the most compelling embedded vision applications use very performance-hungry algorithms. Implementing these algorithms at low cost and low power consumption will require specialized processors. As a result, we expect to see processor suppliers introducing more processors that are optimized for vision applications, and providing more application development support (such as optimized software libraries) for these applications.
Jeff Bier is founder, Embedded Vision Alliance and president, BDTI.
|
|
|
|
|
Navigate to related information
Sanjib.Acharya
3/30/2012 12:15 PM EDT
Something like this is already applied in games, then why not in other day-to-day applications such as switching the TV on by looking at it? More efficient processors and less power hungry algorithms will make electronic vision to be part of our day to day life very soon.
Sign in to Reply
sharps_eng
3/30/2012 2:00 PM EDT
How will this be an advance if it is not accompanied by imagination? Turning on the TV used to take less than a second; I hit the ON switch as I walked past the set, crossing the room from door to sofa. Now it takes longer to give me a picture than my CRT tube TV did. Oh, and now we also have a TV that crashes.
It would be possible to have multiple screens that personalize their output by recognizing individuals as they come and go, OK for surfing (maybe) but that doesn't fix the sharing of audio or the distraction from the other screens. Far more useful is enhancing machine speech recognition using video cues from the face. That can work for smart phones, but not for TVs until zoom and face tracking is added.
I guess it will arrive via the back door like Kinect.
There are possibilities for improving access for the disadvantaged but most sites on the internet are terribly hard work for people of limited vision.
Sign in to Reply
prabhakar_deosthali
3/31/2012 7:27 AM EDT
Electronic vision can be very useful to measure TV viewership especially the commercials . An eye in the TV can monitor the viewer activity and link it to he selected program.
Sign in to Reply
Work to Ride comma Ride to Work
4/2/2012 5:35 PM EDT
George Orwell is rolling over in his grave right now.
Sign in to Reply
wilber_xbox
3/31/2012 12:50 PM EDT
Automobiles safety can be benefited from the new sensors but vision sensors are hard to justify. I do think that consumer applications in which we can use sensory input rather than touch can be huge advantageous.
Sign in to Reply
elctrnx_lyf
4/1/2012 5:19 AM EDT
There would be many more vision processing algorithms would be integrated into the soc's and this will definitely revolutionise the evolution of new in ovative products.
Sign in to Reply
DrQuine
4/1/2012 2:18 PM EDT
First do no harm: prevent false alarms. Everytime I walk through a New York City airport bathroom and hear the "Flushing Cheer" as each toilet in turn flushes, I realize that automation is a double edged sword. There is also no emergency stop switch - so stuck sensors waste staggering volumes of water.
Sign in to Reply
cdhmanning
4/2/2012 5:23 PM EDT
I doubt computer vision can really improve vehicle safety unless it is always augmentative.
It is unfortunately human nature to abdicate responsibility to technology. If a vehicle has a computer vision backup alarm, drivers will soon learn to just reverse blindly until the alarm sounds. When something does go wrong and something is damaged or someone is hurt or killed, the driver's lawyers will be looking for someone to peg the blame on.
Sign in to Reply
Frank Eory
4/3/2012 1:17 PM EDT
One such augmentative machine vision application was demonstrated at this year's CES -- an automotive heads-up display that highlighted (in red) critical objects in the field of vision, like a pedestrian starting to cross the street up ahead.
I think an application like this can definitely improve safety. Drivers will not abdicate their responsibility to look through the windshield, but machine vision just might alert them more quickly to things they otherwise might not notice right away.
Sign in to Reply
Work to Ride comma Ride to Work
4/2/2012 5:31 PM EDT
The Next Killer App: Machines That See; wasn't that Arnold in the Terminator?
Sign in to Reply
selinz
4/3/2012 1:33 PM EDT
Safety, convenience, time saving. Oh, and play, I suppose (for example, all the iPads being sold are really for playing). I must admit that the toilet example by DrQuine struck a nerve. What's worse than the ill timed auto "royal flush?" Our robotically controlled vacuum use more clearly uses automated vision than automated trash lid (dumb sensor) but the latter is no less convenient. Keep it simple s _ _ _ _ _.
Sign in to Reply
Alix
4/3/2012 2:52 PM EDT
I believe that vision in intelligent systems must be properly (intelligently?) integrated with the other system sensory input or it will be next to useless (as referenced by sharps) as the command cascade to action will be a ridiculous trip.
Combined vision with voice recognition, for example, would significantly reduce error in input by linking intuitive gestures with voice commands. You could then point at your TV and say "HBO" and the system would combine the two commands into the proper action. Combo input would nearly eliminate false command input because unless both gesture and statement corresponded there would be no action, and more precise action would be enabled because gestures significatnly reduce ambiguity of intent.
Sign in to Reply