Commentary
Comment
Gaurav123
Very interesting technology! I can see a clear use for close-range ...
Gaurav123
Very interesting technology! I can see a clear use for close-range ...
Using gestures in low power, handheld devices to enhance 2-D touch displays to the third dimension
Klaus Zeyn
2/6/2012 1:48 PM EST
With CES 2012 having been declared by the organizers "the year of the interface," gesture recognition and gesture control have been on full display from a number of vendors at the world’s largest consumer electronics show last month.
This article considers the evolution of the touch-screen interface from two dimensions, to expand to additionally feature a touch-free "third" dimension, where users can also interact with their devices using free-space gestures. While various 3-D gesture sensing technologies are available in the market, a broad acceptance of 3-D user-interfaces in the handheld space still has to happen. The reasons are manifold, with power consumption being a prominent limitation factor. It also presents an overview of an innovative system-on-chip solution that enables gesture control implementation utilizing touch panels ITO (indium tin oxide) layer, and how it can enable a new generation of harmonized touch and non-touch user interfaces.
The natural drive toward 3-D gestures
Touch screens have advanced from relative obscurity in the early 1990s to become the dominant user interface technology in mobile devices today. The industry is constantly reaching for improvements in the resolution of the display, in capacitive 2-D sensing incorporating multi-touch, and the overall computing power of the device. As a result, manufacturers have been able to create a new generation of handheld devices making the internet mobile; however, the success of such products today very much relates to the user interface experience.
It is clear that 2-D gestures have changed the way how we interact with our devices. However, they are reaching their limits in terms of being intuitive and sticky. While touch continues to be an important human modality, we do live in a 3-D world. The most natural interactions for human beings are gestures which happen in three dimensions. Therefore, there is a growing desire, among consumers and device makers, to extend the way of user interaction into the space around devices. A further driver for 3-D gesture sensing is the advent of 3-D displays. When 3-D objects appear in front of a display, the only logical way to interact with the objects is through hand gestures in the same space.
There are many ways in which 3-D gestures can be used to control a device. The market will determine the right balance between intuitive control, very accurate gesture classification, and a large "dictionary" of gestures.
Camera-based architectures have proven to be useful as a 3-D gesture control technology when there is a longer distance between the device and the user. In such cases the viewing angle of the camera and associated blind spot is not a limitation. The significant power consumption required for the processing of camera images makes a constant power supply almost mandatory though. For example, both these conditions are met for the widely embraced Microsoft Kinect—a power consumption of over 1W makes this technology unusable for battery powered devices, though.
For touch-screen centric handhelds such as smartphones or tablets a different approach is best. Developed by IDENT Technology, electrical near-field (e-field) technology can now be used to constantly detect the presence, position and movement of a human hand in the space over a device, i.e. its display.
As a first solution to address the 3-D market the company has designed their GestIC chip as a mixed signal SoC. The GestIC has very little requirements on hardware and power consumption. For optimum power results, the chip can be programmed to automatically switch between a monitoring and an active mode without missing a user’s gesture. As a result it can be left on continuously and does not need to be started explicitly. As an example, for a device used for one hour over the entire day, the resulting current consumption would be below 1mA—well within the power budget of a mobile device.
Of particular interest for touch screens is the fact that the standard ITO layers of the display’s touch panel can be re-used as GestIC electrodes to sense a user’s motion. The ITO layers are used in a multiplexed fashion between GestIC and standard touch controller IC’s. This allows sufficient data points for the GestIC operation that do not interfere with traditional touch sensing functionalities. Also with re-use of the ITO layers, the technology is invisible to the user’s eye—contrary to surface cuts required for other 3-D gesture sensors— while no extra costs are added.
The usage of ITO layers as a sensor zone allows 100 percent surface coverage without any detection blind spots—unlike camera systems in close range. Although the technology shares some characteristics with standard capacitive sensing, there are significant technology differences leading to a detection range for GestIC of up to 20 centimeters above the device.
A range of additional system benefits derive from the physical principal of e-fields. Electrical fields are widely independent against changing environmental conditions like sound, lighting or body texture. Robustness is obtained through the used carrier frequency in the range of 100 kHz only, which results in a quasi static field at low RF interference against common wireless technologies. In addition to that, the GestIC chip runs sophisticated algorithms for noise reduction and adaptive calibration. A high resistance against charger noise and changes of the earth grounding conditions are being realized.
This article considers the evolution of the touch-screen interface from two dimensions, to expand to additionally feature a touch-free "third" dimension, where users can also interact with their devices using free-space gestures. While various 3-D gesture sensing technologies are available in the market, a broad acceptance of 3-D user-interfaces in the handheld space still has to happen. The reasons are manifold, with power consumption being a prominent limitation factor. It also presents an overview of an innovative system-on-chip solution that enables gesture control implementation utilizing touch panels ITO (indium tin oxide) layer, and how it can enable a new generation of harmonized touch and non-touch user interfaces.
The natural drive toward 3-D gestures
Touch screens have advanced from relative obscurity in the early 1990s to become the dominant user interface technology in mobile devices today. The industry is constantly reaching for improvements in the resolution of the display, in capacitive 2-D sensing incorporating multi-touch, and the overall computing power of the device. As a result, manufacturers have been able to create a new generation of handheld devices making the internet mobile; however, the success of such products today very much relates to the user interface experience.
It is clear that 2-D gestures have changed the way how we interact with our devices. However, they are reaching their limits in terms of being intuitive and sticky. While touch continues to be an important human modality, we do live in a 3-D world. The most natural interactions for human beings are gestures which happen in three dimensions. Therefore, there is a growing desire, among consumers and device makers, to extend the way of user interaction into the space around devices. A further driver for 3-D gesture sensing is the advent of 3-D displays. When 3-D objects appear in front of a display, the only logical way to interact with the objects is through hand gestures in the same space.
There are many ways in which 3-D gestures can be used to control a device. The market will determine the right balance between intuitive control, very accurate gesture classification, and a large "dictionary" of gestures.
Camera-based architectures have proven to be useful as a 3-D gesture control technology when there is a longer distance between the device and the user. In such cases the viewing angle of the camera and associated blind spot is not a limitation. The significant power consumption required for the processing of camera images makes a constant power supply almost mandatory though. For example, both these conditions are met for the widely embraced Microsoft Kinect—a power consumption of over 1W makes this technology unusable for battery powered devices, though.
For touch-screen centric handhelds such as smartphones or tablets a different approach is best. Developed by IDENT Technology, electrical near-field (e-field) technology can now be used to constantly detect the presence, position and movement of a human hand in the space over a device, i.e. its display.
As a first solution to address the 3-D market the company has designed their GestIC chip as a mixed signal SoC. The GestIC has very little requirements on hardware and power consumption. For optimum power results, the chip can be programmed to automatically switch between a monitoring and an active mode without missing a user’s gesture. As a result it can be left on continuously and does not need to be started explicitly. As an example, for a device used for one hour over the entire day, the resulting current consumption would be below 1mA—well within the power budget of a mobile device.
Of particular interest for touch screens is the fact that the standard ITO layers of the display’s touch panel can be re-used as GestIC electrodes to sense a user’s motion. The ITO layers are used in a multiplexed fashion between GestIC and standard touch controller IC’s. This allows sufficient data points for the GestIC operation that do not interfere with traditional touch sensing functionalities. Also with re-use of the ITO layers, the technology is invisible to the user’s eye—contrary to surface cuts required for other 3-D gesture sensors— while no extra costs are added.
The usage of ITO layers as a sensor zone allows 100 percent surface coverage without any detection blind spots—unlike camera systems in close range. Although the technology shares some characteristics with standard capacitive sensing, there are significant technology differences leading to a detection range for GestIC of up to 20 centimeters above the device.
A range of additional system benefits derive from the physical principal of e-fields. Electrical fields are widely independent against changing environmental conditions like sound, lighting or body texture. Robustness is obtained through the used carrier frequency in the range of 100 kHz only, which results in a quasi static field at low RF interference against common wireless technologies. In addition to that, the GestIC chip runs sophisticated algorithms for noise reduction and adaptive calibration. A high resistance against charger noise and changes of the earth grounding conditions are being realized.
|
|
|
|
|
Navigate to related information
Gaurav123
2/7/2012 11:57 AM EST
Very interesting technology! I can see a clear use for close-range gesture/proximity detection specifically when a RGB camera has limited field-of-view. What is the resolution of detection? Can it detect individual fingers?
The only drawback I see is that this solution will require a HW change as compared to a RGB camera based solution, which would only need a SW update (given that most of the handheld devices now have a RGB camera). Are you seeing a willingness in manufacturers and consumers for this approach?
Sign in to Reply
Gaurav123
2/7/2012 11:58 AM EST
Very interesting technology! I can see a clear use for close-range gesture/proximity detection specifically when a RGB camera has limited field-of-view. What is the resolution of detection? Can it detect individual fingers?
The only drawback I see is that this solution will require a HW change as compared to a RGB camera based solution, which would only need a SW update (given that most of the handheld devices now have a RGB camera). Are you seeing a willingness in manufacturers and consumers for this approach?
-Gaurav , Texas Instruments
Sign in to Reply