Design Article
Inside Nethra's NI-20x0 video and image processor for handsets and digital cameras
Murty Bhavana, Engineer
and VP of Marketing,
Nethra Imaging Inc.
7/6/2005 12:02 AM EDT
System designers need real-time video and image processing in a small form factor while consuming low power. This challenges IC designers to develop solutions that meet these extreme system design requirements, as each of these requirements tends to drive the solution in the opposite direction. For example, a high performance IC design will drive the cost and power to a high number. Solutions developed for these markets tend to be balanced towards one of the parameters more than the other.
System Block Diagram of a Camera Application
To determine the design challenges let's begin by looking at the system design requirements. The typical digital image/video capture system will have optics, image sensor (CCD or CMOS), analog front end, timing generation, "raw" image processing (taking raw image sensor with Bayer formatted color filters and processing all the way to YUV or YCbCr formats), and still or video compression engine (if required). CPU may be implemented at a system level to give the system designer some flexibility of system level software to get a better control of how the camera operates. In addition system level control functions are required via the support of PWMs, GPIOs, I2C and/or SPI interfaces to control other system level mechanics.
Nethra's NI-20x0 simplifies this system level design by integrating most of these system level functions into a SOC in an 8 mm x 8mm package. The device includes the following functionality on the same die:
___o Debugging via JTAG or debug UART
This high level of integration in a small form factor has been implemented without sacrificing performance requirements.
Figure 1. NI-20x0 detailed block diagram showing various functional blocks.
Color Processing
One of the biggest concerns of video/imaging system designers is the quality of algorithms they want to implement. This is one of the single biggest reasons for adopting DSP or FPGA-based architectures for camera designs. High volume image processing applications are driven toward mainstream DSPs from Texas Instruments, and for niche applications designers tend to use FPGAs in their designs.
Nethra's color processing algorithms are implemented using ASIC design methodology. All the memory required during the color processing is implemented as pipeline memory and the color processing algorithm hardware is tightly coupled to this memory. This design implementation permits the operation of the pixel processing at 54 million pixels per second.
The device family is capable of taking the raw image sensor data (ADC output from the CMOS image sensor or the Analog Front End in case of CCD image sensors); perform the entire image processing steps (listed below) and send CCIR-656 data out of the chip.
The products support following image processing algorithms:
___o Enhanced Low Light Processing
___o Electronic Image Stabilization
___o Red Eye Detection in Hardware
___o Auto Focus
___o Auto Exposure
___o Auto White Balance
The color processing algorithms have the potential to produce print-quality still images or crisp, vivid video depending on the system requirements. Histogram and statistics data are collected from image frames using multiple zones while the image data is flowing through the pipeline. This data is analyzed and used for implementing the camera system algorithms (AE, AF and AWB) to control sensors, motors, mechanics of the camera system, and image pipeline parameters.
Adaptive Light Processing
All imaging devices face the problem of having to handle image scenes that tend to have extreme lighting conditions at the same time (high intensity in one region of the image and low intensity in another region). Any auto exposure algorithm will try to adapt to one area or the other depending on the implementation (area vs. spot metered). NI-20x0 family of image processors supports compensation for such extreme lighting exposures in the same scene by implementing an adaptive lighting algorithm.
Adaptive light processing is implemented to support two modes of operation " backlit and normal lit scenes. Histogram and statistics engine are used to determine the profile of the image scene. This information determines the necessary adjustments to be made in the Luma and/or chroma so that the resulting image achieves a desired profile. The adaptive light logic then programs the optimum image adaptive adjustments to the hardware, which will perform the computation at very high speed. The following image shows the advantage of such an algorithm in capturing images in extreme lighting conditions:
Figure 2. Adaptive lighting algorithm brings in details (top image) which may otherwise be hard to capture (bottom image).
