Design Automation

Functional Verification of Graphics Designs

Graphics subsystem design verification is complicated by visual quality ambiguity.

by Thomas L. Anderson, Michael Sleator, and Douglas Stiles

Complex graphics systems are representative of the upper end of today's product design spectrum. Projects requiring full-color, 3D graphics commonly push designs to a half-million gates or more. The challenge in verifying designs of this size is on a par with many microprocessors and full-scale computers. Graphics systems have an added complexity, separating them from microprocessor designs: their arithmetic calculations need only be precise enough to generate images pleasing to the human eye. This aspect of verification affects the process in several ways.

Given the complexity of graphics systems, the design and verification of hardware and software together has been a major thrust. The phrase "hardware/software co-design" has become something of a cliché for EDA tool vendors. Nonetheless, it is a desirable goal, especially for graphics and other design domains in which implementation can take place in software for embedded processors, in software running on a host computer, or in hardware.

To achieve this goal requires simulating as much software as possible before any of the chip designs are committed to silicon. This provides better hardware design verification and allows most software to work as soon as the system is powered on. One useful strategy is for the software group to verify any embedded processor code on an instruction set emulator that operates with high-level graphics architectural models. Additional system components, such as memory, must also be modeled to emulate as much of the system operation environment as possible (see Figure 1 ).

Models for this "architectural simulation" phase are written in a high-level language such as C and at a more abstract level than the RTL models required for logic synthesis.

For example, the models might correctly represent the number of clock cycles needed to complete a particular operation but not the precise state of the design during the intermediate cycles. The architectural simulator may not match the final design at the individual graphics operation level. Thus, it is important to have points in the simulator at which data can be captured and later compared with RTL simulations. For example, the contents of the frame buffer models can be compared after rendering a complete scene, rather than comparing the results of each operation in the render process.

If a model of the embedded processor is available in the RTL functional simulation environment, the same embedded code can be run on both simulators. In fact, the same code may even be run on the final system, although some low-level routines may have to be different. A conditional compilation construct (such as C's "#ifdef") is often used to allow the same source file to be compiled for the three different functions.

Fully accurate processor models are generally slow and hard to obtain; the functional simulation environment often uses a hardware modeler to address this problem. This modeler contains the actual processor chip to be used in the final design. The rest of the design running in the functional simulator provides the modeler its stimulus and accepts the modeler's results.

The types of code running on embedded processors typically include core graphics primitives, communication programs, synchronization routines, and hardware diagnostics.

A hardware modeler may also be used for memory and other complex chips and macros that are difficult or inefficient to model in RTL form. However, there is one disadvantage to this approach. When the entire design is modeled in C or RTL code, it is easy to directly "poke" a value into any part of the model or "peek" at any current value. This ability to "poke" and "peek" can provide a means to bypass some uninteresting tasks, such as loading a program into an embedded processor's internal memory prior to running the desired test. When the actual part is used in a hardware modeler, it is usually necessary to go through the full load sequence. This may be a significant portion of the runtime for each verification test.

A hardware modeler can even be used to test the implementation of a portion of the final design. Part of the design can be synthesized from RTL into an FPGA or ASIC prototype. The resulting chip can then be run on the modeler in conjunction with the C or RTL simulations for the remainder of the design. Most of the ideas discussed in the article may be employed in a "mix and match" configuration (see Figure 2 ).

Figure 1. A typical EDA flow for a complex chip-based design project. It shows only those design tools used for the verification of chips--multi-chip projects would also make use of tools for schematic capture, analysis, and route of any printed-circuit boards or multi-chip modules required.

Verification stimulus In most graphics designs, the actual graphics subsystem is driven by a host processor that provides the remaining system functions. Some sort of command stream is generally sent from the host processor to direct graphics subsystem operation. In the architectural and RTL verification environments, three approaches are often used to provide this command stream. The simplest of the three is to run the host software and direct its command stream to an interprocess communication channel, such as a UNIX socket, that drives the verification environment.

Since the architectural simulation models are written in a high-level language, it is easy to connect them to this socket. Connecting to the RTL simulator may be more difficult. Not all simulators provide a mechanism linking together high-level language utilities and the design's RTL description. A commonly used mechanism is the Programming Language Interface (PLI) that is supported by most Verilog language simulators.

It may be impractical to directly connect the host software and the verification environment. For example, the host software may run on a hardware platform much different from the computers on which the EDA tools operate. It is also quite common that the software and hardware are being co-developed and need to have some independent verification prior to integrating them. In such a case, it may be possible to capture a command stream from an existing system that runs a similar graphics protocol. This may be accomplished by redirecting the command stream to a file rather than to the graphics subsystem, or by using data acquisition instruments to capture the commands being sent to the graphics subsystem.

Once a captured graphics command stream is available in a file, it is a simple matter to write some code to read these commands into the verification environments as stimulus. This technique works best with unidirectional commands. Commands that involve extensive handshaking, such as the read back of frame buffer contents to the host, are best tested with host software that directly interacts with the verification tools.

Regardless of how many command streams are captured and executed, it is also necessary to generate some hand-coded tests. These tests may test edge cases that are unlikely to occur in the captured command streams or more completely exercise the design to increase the fault coverage provided by test vectors derived from verification. Such tests may consist of hand-written or programmatically generated command sequences or may "poke" at lower levels inside the verification models.

Verification results Determination of test success or failure is a necessary part of any verification process. Because of the nature of graphics designs, often the final verification results are best evaluated by examining an image on the screen. Although designers use analytic techniques to determine accuracy and predict image quality, what really matters is how the graphical images appear to the human eye.

Figure 2. Some of the ways in which the different pieces of the verification puzzle can be combined. Few projects actually use all the possible combinations. Keeping connections among the components during the design process can be very valuable. For instance, if it's possible to ensure that frame buffer contents should be the same for both the architectural and RTL simulations, then the files may be compared without the need to examine images for runs of the same test in the two verification environments.

Most graphics verification tests generate and store some sort of image in the model of the frame buffer. It is common for verification tools to save the frame buffer model image in a file. The tools can convert the image so it can be loaded into the frame buffer of a graphics system for display. Once an image has been found satisfactory, its corresponding frame buffer contents can be saved as a known-good or "golden" image and used for comparison in future runs of the verification test.

It is desirable to be able to do bit-for-bit comparisons with "golden" images so comparisons may be easily automated as part of the verification process. This requires that the architectural model uses exactly the same bit resolutions and arithmetic algorithms as the RTL implementation. If not possible, variations in frame buffer contents may produce two images with no differences visible to the human eye. Doing an inexact comparison, rather than a bit-for-bit comparison, demands a heuristic approach in which some forms of difference are considered to be acceptable.

The "golden" image approach usually works well for verification tests driven by, or derived from, graphics command streams. Hand-coded tests may need to "peek" inside the verification models to check for expected results or to ensure the absence of error conditions. Additionally, it is quite common to include monitors--routines that run as part of the verification process and check for erroneous behavior. For example, a monitor might check a parity error signal and report a verification failure for a test that generated a parity error without intentionally testing this condition.

There are also some effects that may be visible only when a series of graphical images is displayed. For example, a frame-by-frame examination of an animated sequence provides only a limited sense of what the final result will be. For this reason, graphics designers play multiple graphics images on a display screen as close as possible to actual speed. If a writable optical disk is available, full-speed playback of the sequence is achievable.

The role of emulation Hardware emulation has become increasingly popular in the last few years and has been used successfully on a number of complex graphics design projects. Emulation entails mapping the system under design into a reconfigurable hardware platform built from FPGA or similar programmable devices. Emulators typically operate at a speed far less than the final design. Mapping complex components or ASIC macros into the FPGA devices is difficult and presents the main challenge of emulation.

Graphics emulation projects usually address these issues by emulating the design only up to the frame buffer. A special frame buffer built to drive display screens at standard rates is loaded at a much slower emulation speed. This allows the direct viewing of generated images without the need to convert a frame buffer model file to a display form.

Any embedded processors and memories in the graphics subsystem are usually not modeled in FPGA form. Instead, commercial emulation systems accept circuit boards containing actual chips, such as processor and memories. The connection between the emulated design and the actual chips is analogous to the connection between RTL simulation and hardware modelers.

Because it takes considerable time to map a design into emulation hardware, designers do not start emulation until they have a number of tests working in the RTL verification environment. It is common to run hand-coded tests and short graphics command streams in RTL, turning to the much faster emulator for long command sequences. One additional difficulty is that the host processor generating the commands may need to be slowed to emulation speed, unless its interface to the graphics subsystem can accommodate widely disparate clock speeds.

The goal of all these verification approaches is to co-develop and co-verify hardware and software so that everything runs together when the prototypes are built. It is a rare project, graphics or otherwise, that achieves this goal completely. The long-standing rule of prototype bringup is "if it wasn't simulated, it won't work." The better the verification strategy, the more realistic and extensive the testing can be and, therefore, the more that will work perfectly when the prototypes arrive.

Thomas L. Anderson is a freelance author and vice-president of Engineering at Virtual Chips Inc. (San Jose, CA). He has held a number of design and EDA management positions at AMD, Kubota Graphics, and Stardent.

Michael Sleator is a freelance author and senior systems engineer at Artemis Research (Palo Alto, CA). He has a diverse background in developing graphics hardware and software at Radius, Kubota Graphics, and Stardent.

Douglas Stiles is a freelance author and the manager of Design Automation and System Verification at the 3DO Company (Redwood City, CA). He has been a key contributor to several generations of video game products.

To voice an opinion on this or any Integrated System Design article, please e-mail your message to michael@asic.com.

integrated system design  May 1996