3-D gesture control breaks out of the game box

Hillcrest founder and CEO Dan Simpkins proclaimed 2011 “the year of the smart TV,” adding, “For the first time in more than 50 years, a new input technology has come to the market for television.” Hillcrest’s Loop pointer, an in-air mouse designed for consumers who connect their computers to a television, uses its Kylo Browser for IPTV.

The Magic Motion remote that LG showed at CES uses Hillcrest’s Freespace gesture recognition technology to let users navigate complex point-and-click on-screen interfaces for Web-based and conventional TV content. 

Competing approaches include Philips’ uWand. For its handheld controller, Philips opted for an integrated infrared camera that senses IR beacons in the TV to enable accurate motion tracking without requiring a gyroscope. Most other IPTV remotes, however, use MEMS gyros.

Movea, for instance, announced at CES that laptop keyboard vendor Sunrex Corp. (Taiwan) would have controllers later this year that would use Movea’s MEMS-based MotionIC platform and SmartMotion technology for 3-D gesture recognition.

“The next generation of motion remotes will recognize all sorts of new gestures,” said Dave Rothenberg, worldwide marketing manager for Movea. “For example, Mom and Dad will be able to unlock adult content on the TV by waving their signatures in the air, whereas when the kids come in the room and do the same thing, the parental controls will be activated.”

Microsoft’s Kinect, meanwhile, challenges the conventional wisdom by moving the gesture-sensing hardware, which includes a MEMS accelerometer, out of users’ hands and into the head unit. Microsoft developed its own 3-D recognition algorithms for Kinect based on optical recognition technology licensed from GestureTek.

Kinect classifies gestures within the strict confines of actions in a particular game, such as virtual volleyball. The technology segments images by projecting a regular array of infrared dots onto the player with a laser, then measures the reflected intensity of each dot. Less intensely reflected dots are assumed to be reflected from the background; more intense dots are assumed to come from the user in the foreground. Kinect then animates an avatar with its best guess of the user’s actions. A MEMS accelerometer from Kionix helps aim the cameras at the user more accurately.

The technique sacrifices some accuracy in exchange for the user’s mobility, according to analysts. “I do not believe that the camera-based recognition system from Microsoft is accurate enough to satisfy many gamers, who will probably want to continue holding the controller, making the Sony Move a better candidate for hard-core gamers,” said iSuppli senior analyst Jérémie Bouchaud.

“Microsoft’s solution works for the audience it is targeting: families that want to jump in and out of a game quickly and want an easy and immediate experience,” said Piers Harding-Rolls, head of games at IHS Screen Digest. “Sony’s Move, on the other hand, is a hybrid solution, using sensors to track motion and a camera to track position. At this stage, Sony’s theory is to target enthusiast gamers with more accurate sensor technology.”

Camera-based technology like Kinect’s, said Gartner analyst Jim Tully, “is not the endgame for gesture recognition. It has a place, but accelerometer-gyro combos also have a place.

“For instance, the Kinect camera can’t detect complex movements in a multiuser situation when one user is blocked by another user. It is also not so good when a user turns his back on the camera . . . These situations would need multiple cameras, [which would not be] very feasible in most situations.”

But GestureTek, whose technology already tracks the 3-D motion of millions of cell phones by observing a changing camera image—claims that optical gesture recognition will eventually outperform MEMS-based devices.

“Today, the resolution and accuracy of optical gesture recognition are not as good as when using MEM inertial sensors, but they’re good enough for most games,” said Vincent John Vincent, co-founder and president of GestureTek. “And as camera resolution gets better, we believe optical gesture recognition will eventually surpass MEMS by enabling devices to track the movement of every part of your body, with pixel-level accuracy.”

An actor (center) wearing Xsens' MVN Motion Capture lycra suit - which is studded with MEMS inertial sensors from Analog Devices - mimics the pose of a character in a Marvel Comics "Iron Man" illustration (l.) as the basis for creating an animation for Paramount Pictures' "Iron Man 2" (r.). SOURCE (l.-r.): Marvel, Xsens, Paramount

Philips' uWand technology aims for the same applications as MEMS-equipped controllers but does not use MEMS devices. Instead, it integrates an infrared camera to sense IR beacons from the TV, thus enabling motion tracking without a gyroscope. SOURCE: Philips