Modern digital cameras and cell phones containing digital cameras are merely complex microprocessor systems with ever-increasing amounts of functionality. This increased functionality is spread out between the software portion of the system and several processors. The CMOS image sensors typically have extensive image processing functionality built into them to help manage tasks like cropping and compression. These operations, however, are often duplicated in the hardware, which leads us to ask, why is this functionality duplicated and which IC performs these functions?

To understand the operation of a product, traditional circuit reverse engineering can be used; however, it can be an expensive undertaking. Software reverse engineering is another option; but this can be time-consuming work. So when circuit and software reverse engineering are not possible because of time and budget constraints, we turn to functional testing.

The timing picture of this particular smartphone consists of "power on" initialization; viewfinder mode; image capture;display off between shots; and return to the viewfinder mode. Click on image to enlarge.

Functional testing techniques can be used to demonstrate that a particular method of operation is being used. All examples featured in this article are related to image processing devices; however functional testing has been successfully used on other devices including TVs, Mobile Internet Devices (MIDs), computers, etc. Our objective for this article was to demonstrate how system functional testing techniques can be used to investigate the inner workings of an image processing device. The results of our experiments were had within just a few days and at a minimum expense.

The three scenarios we investigated include: testing a device using a datasheet of the camera module; testing a camera module as a black box; and the last case study requires testing different signals of the control logic of an image sensor on a board.

The first consumer device analyzed was a smartphone with a built-in 1600 x 1200 active pixels digital camera. The camera contains a serial control interface, JPEG encoder, and supports different output formats including RGB, YUV, Bayer and JPEG. Our goal was to understand the various modes of operation of the camera module and to find out where JPEG compression occurred. The image module contained an internal JPEG encoder. Therefore the compression process could either take place inside the sensor or in a separate IC.

We tested the device by capturing signals from the camera module with a logic analyzer, followed by a software-based reconstruction of commands on a control bus and determination of the image data format. Figure 1 shows the big picture, the operation of the phone during image capture.

The camera module has two preliminary programmed contexts that have different parameters. Both contexts are set up during the "power on" initialization. During operation it requires several commands from the control bus to switch between the contexts; the use of a slow speed serial control bus greatly reduces the switching time between image shots. Context_0 with 640x480 pixels resolution and RGB565 output format is used in the viewfinder mode and context_1 has user-selected resolution and a JPEG 4:2:2 output signals format.

To view the image capture mode with 1600 x 1200 pixels resolution, and the file containing values of data output signals from the logic analyzer and a fragment of the command file on the serial control bus of the camera module, click here. (Note the start of the frame with JPEG defined pattern FFD8_FFDB).

As a result of the system testing we were able to successfully determine the different modes of operation and identify where the JPEG compression took place on the SmartPhone.

In the second camera example, our goal was to analyze output data from the camera and find out if the image data is decimated in the camera module, or if the camera sent raw data and that data was decimated by the image processor. In this case we didn't have a camera module datasheet; this added an additional layer of complexity to the test.

Digital camera test setup gave access to the main board and the image sensor. The challenge was to keep the camera fully functional after this process was complete. We found the required clock and synchronization signals from the image sensor to the main board and used these as test points to connect to the logic analyzer.

We connected a logic analyzer to all signals of the sensor module and figured out the control, vertical (Frame) and horizontal (Line_valid) synchronization, and data bus signals. Counting the number of horizontal pulses during a valid frame signal helped to answer the question about decimation in capture and viewfinder modes. If the decimation process is done outside of the camera module, the raw data will not change at the output from the camera module. We saw the opposite: the number of horizontal pulses within the active frame changed depending of a selected resolution. The testing allowed us to successfully investigate the decimation process in a phone without having to conduct software or hardware reverse engineering.

To view an image of the viewfinder and capture modes with active frame view, click here. In Viewfinder mode, the resolution is 640 x 480 and the number of lines in an active frame is 480, and in the Image Capture mode, the resolution is 1600 x 1200 where the number of lines in a frame is 1204.

In our final digital camera example we investigated the "window readout" feature of a camera. The camera contains a large 13.1 megapixel CMOS sensor that could not be fully read out during zooming, panning or aspect ratio changes. The goal was to determine if these features use a change in the vertical and horizontal addressing in response to pan, zoom, or aspect ratio change commands. The testing required the measurement of the synchronization signals. If the vertical and horizontal addressing circuits were used, the active frame and number of line pulses would change with zoom and aspect ratio changes. If there were no changes, then the cropping of the image occurred outside of the sensor module. The test results showed that the same image sensor area has been read for both aspect ratios 3:2 and 16:9 and is not dependent on the zoom and pan commands, i.e. the frame period and a number of lines in each frame stayed the same regardless of incoming commands.

To view the timing diagram for aspect ratio 3:2 and the timing diagram for aspect ratio 16:9, click here.

In all three examples we see that the division of the image processing operations depends on the product, however it would be fair to say that the full functionality of the image sensor was not utilized in any of the tests we conducted. Perhaps this is because the designers of these devices wanted to limit image processing operations to the main image processor IC.

The above examples show the effectiveness of system functional test analysis. With a minimum knowledge of device functionality, with no datasheet and pin definition, we were able to find answers about specific operations within a reasonable timeframe and cost.

Eduard Golod is Senior Analyst at Semiconductor Insights. Golod has over twenty years of experience in telecommunications, microprocessor architecture, and system interconnect. Prior to joining SI, he held ASIC, FPGA, and system design positions at Nortel, Tundra and AMD. Golod received his M.S. degree in Electronic and Computer Engineering from Belarusian State University.