The mobile industry originally assumed small displays would be mostly used “for simple user interfaces, webs and games,” said Tom McMahon, video expert and founder of The Del Rey Consultancy. But that’s no longer true. Devices are coming out with higher resolution displays, and consumers are watching more video on tablets than anyone in the mobile industry originally expected.
Coupling this industry-wide trend [for higher-resolution mobile displays] with consumers’ craving UHDTV, the mobile industry “needs to figure out a way to do display processing, or signal processing, either in a box [tablet] or in a head-end from which video is being distributed, McMahon noted.
DisplaySearch’s Shim acknowledged that display processing is one solution that could improve the video playback experience. “The challenge is that as far as I know there aren’t any display processors available in volume specifically for tablets,” he said. “Another challenge would be cost. As tablet average selling prices continue to fall, adding new components will be difficult.”
McMahon agreed. “There is cost for power, and the cost for silicon. Something’s got to give. You need to pay for it.”
Bruce Walicek, president and CEO at Pixelworks, is one executive willing to discuss how display processing technology is becoming even more important on mobile devices.
In the company’s earning call earlier this year, Walicek noted a “gigantic tsunami of pixels that are headed for the entire ecosystem” and talked of Pixelworks’ display processing technology, now powering many of the tier-one TV manufacturers’ Ultra High Definition TVs. During the call, he said, “Higher resolution of course means more noticeable video quality problems, and video quality problems associated with higher-resolution larger screens are now migrating to small ones.” Although he stopped short of disclosing his company’s product plan, Pixelworks appears focused on developing a display processor specifically designed for mobile screens.
Of course, “there are a billion different ways to up convert video,” said McMahon. McMahon, who used to work for Microsoft and Broadcom, explained, “Whether it is a motion estimation, motion vector or motion protection, there are different ways to estimate--or guessing--where the pixel goes next. It’s an interpolation over time.” A company like Farudja Labs, later acquired by ST Microelectronics, worked for years on the development of video processing algorithms and products that powered large-screen TVs, he added.
Bringing the level of sophisticated display processing currently used in a large-screen UHDTV to mobile devices is a stretch. The issue of cost is compounded by the impact on battery life. Beyond an external display processor chip, this could require a much more holistic system-level approach.
Noting that today’s tablets consist of “pretty much one big system-on-chip and battery,” McMahon speculated that vendors such as Qualcomm and Broadcom are likely working to integrate some level of display processing.
Just to be clear, if you are streaming videos onto your tablet from Netflix, for example, you need a good connection on WiFi or 4G LTE. But you'll be streaming 1080p at best--which even then is a slightly lower resolution than the iPad Retina screen. Obviously, with a less-than-optimal connection, video quality continues to decline. But what happens in video streaming or even in video chat is that the frame rate is often reduced, which results in judder. If a new mobile display processor is capable of frame-rate conversion, it can, in theory, recover the original frame rate, making it smooth again.
Display processing for video is fairly mature, so it's a wonder that it hasn't already been integrated into the "one big system-on-chip" that is already decoding the video and doing nearly everything else in the tablet.
Yes there are costs in silicon area and power consumption, but as tablets get increasingly used to watch video, consumers will demand the same level of video quality they get from their HDTV sets. This gaping hole in tablet performance will not remain unfilled for very long.
This is the same hill that digital cameras and printers climbed more than a decade ago. Many current printers can do 1440 dpi w/o popping a sweat. This, coupled with the extreme resolution of high end cameras makes pixelation in a printed image either a blunder on the part of the user or an intentional effect.
But isn't video quality on newer mobile devices more an issue with the source than the display? There's only so much information in an image. You can fix some artifacts and clean up pixelation but eventually you hit a limit - that's when you need a higher resolution source to match your high resolution display - and I'm sure device makers will be happy to accommodate this.
The maturity shows in the intellectual property related to video processing. It is essentially impossible to implement video processing without numerous licensing deals. Everything down to the very idea of digital video processing is under patent protection in most countries. Thus the HTML5 video tag debacle. I agree there is no technical issue preventing it, but there are substantial legal and cost issues. So I'm not so sure the gaping hole will be quickly filled.
Qualcomm acquired HQV video processing assets from IDT over a year ago. HQV video processing has been incorporated into high end consumer products such as Onkyo home theater receivers. Does Qualcomm have plans to integrate HQV into their mobile SoCs?
Interesting article and fully in-line with my view.
Relevant to note is that we can already TODAY offer end-users the proper experience on existing (high end) mobile devices and platforms by applying advanced video algorithms in software. In contrast to (traditional) TVs, mobile devices have evolved to very sophisticated platforms capable to run such solutions in software by smartly leveraging the existing hardware acceleration available in these platforms.
For instance on an iPad 3rd or 4rd generation (with retina display), this is for instance offered by the "SuperSharp" feature in CineXPlayer HD ( https://itunes.apple.com/app/cinexplayer-hd-best-way-to/id384098375?mt=8 ).
For more information on these gap-closing video enhancements for mobile devices (Android and iOS), have a look at our website or contact us:
Join our online Radio Show on Friday 11th July starting at 2:00pm Eastern, when EETimes editor of all things fun and interesting, Max Maxfield, and embedded systems expert, Jack Ganssle, will debate as to just what is, and is not, and embedded system.