Video encoding, SoC development, and TI's DSP architecture

H.264 provides similar visual quality to the MPEG2 video standard at half the bit rate. However, the H.264 video algorithm is highly complex compared to MPEG2 in terms of computation complexity as well as data bandwidth requirement. In fact, H.264 encoding and decoding is so complex it is not uncommon to benchmark a given SoC by its capability to support high definition encoding /decoding of video content in H.264 video format.

This article provides a brief overview of the basic video encoder and challenges and considerations to provide the optimal video encoding solution for a given application segment, including choosing the right SoC architecture, software challenges, visual quality tradeoffs, scalability, ease of integration and time to market.

Tuning the encoder for each application
High definition video encoding applications (720p, 1080i) are growing rapidly in the consumer market space. The application spectrum ranges from low to high-end applications including digital still cameras, surveillance, videophone, and digital TV broadcasting. The overall system requirements, acceptable limitations and various tradeoffs across these applications vary a lot.

The main parameters for considerations are the visual quality, cost of solution, ability to support multiple video formats, power consumption and communications delay.

Let’s consider a few examples: For the Broadcast market, video quality is the main consideration, while the Digital camera market is very cost sensitive for each unit sold. Ability to support multiple video formats is a primary requirement for the IP set-top box market, while delay is one of important consideration for video conferencing. The list just goes on. The video encoder solution has to be tuned based on application constraints for optimal performance.

Complexity challenges
Video algorithms require tremendous amount of computation power and data bandwidth. This complexity depends on encoding vs. decoding mode, video standard, resolution, frame-rate and visual quality constraints. Typically encoding is more complex than decoding by a factor of about 3-5. The complexity scales directly with resolution and frame-rate requirement. High definition encoding (e.g. 720p, 1080i @ 30 fps) with H.264 format is the most difficult scenario for all aspects of SoC and product development.

High visual quality requirement for encoding needs sophisticated set of algorithms, which comes at the expense of computation complexity and data bandwidth. Typically H.264 HD encoding will need data bandwidth in the range of 2-3 Gbits/second for 720p@30 fps. Also, computation power requirement will easily go in the range of GFLOPS based on actual encoding algorithm. This puts tremendous pressure of on SoC architecture & design as well as software optimizations to enable the HD encoding solution.

Choosing architectures
The video standards (H.264, MPEG) are getting better in terms of compression efficiency, and there are a host of other proprietary standards that given silicon has to support based on target application e.g. Real Video, Window Media, On2 video etc. All these dynamics will require programmable solutions to keep pace with arrival of new standards and features. The performance requirements in terms of frame rate, resolution and visual quality are going northwards day-by-day. Traditionally ASIC solutions are used to meet higher performance requirements, but they lack programmability to address dynamics in video market space. For the consumer market, cost is one of the major criteria which determines the success of given silicon. In mobile space, the power consumption is one of the most important criteria as it determines battery life. The following table summarizes various SoC architecture tradeoffs.

Table 1: SoC table

Texas Instruments DSP and Co-Processor
The TMS320C64x+ family is fully programmable DSP designed to cater to video applications. The C64x/C64x+ DSP core enables high-end video applications for set-top box, personal video recorder, high-end cell phones etc. The need to support higher performance can be catered to by increasing clock frequency or using multiple DSPs. The disadvantage of these approaches is increased silicon area (hence the cost) and high power requirement.

The other alternative is the addition of a special purpose co-processor to improve imaging and video performance at lower cost and power. This approach has advantages of both worlds i.e. full programmable DSP as well as ASIC solutions. The co-processors are used to boost video and imaging performance by providing more processing power and special purpose dedicated hardware (e.g. support of VLC/VLD in hardware). These co-processors operate at different clock speed and require lesser silicon area. The video and imaging applications using co-processors will consume less power. DavinciTM from Texas Instruments falls into this category.

Introduction to the video encoder
Fig. 1 shows a block diagram of a generic video encoder. Motion-estimation is used to find motion of macro-blocks using motion vectors to reduce temporal redundancies among input frames. Later, transform (mostly Discrete Cosine Transform: DCT) is performed on the motion-compensated prediction difference frames for de-correlation of prediction error. The prediction error is later quantized as per input bit-rate requirements. The quantized DCT coefficients, motion vectors, and side information are entropy coded using variable length codes (VLC’s). The reconstruction path in encoder consists of inverse transform, quantization, loop filter and motion compensation to mimic operation on decoder side.

View full size
Figure 1: Generic video encoder block diagram

The various existing video standards follow the same block diagram, but differ in flexibility and details within each block. The following table summarizes the difference between MPEG2 main profile (MP), MPEG4 Advanced simple profile (ASP) and H.264 high profile (HP).

Table 2: MPEG2, MPEG4, H.264 table

Next: Typical Video SoC Architecture