Testbench Tutorial, Part II This column concludes Larry Saunders and Yatin Trivedi's discussion of testbenches, begun last month. By Larry Saunders and Yatin Trivedi

In the first part of this two-part column (published last month), we discussed self-checking testbenches and talked about the uses of randomly generated test patterns. In this, the second part of the column, we will continue our review of verification approaches covering the uses of file-based and algorithmically based test pattern generation techniques, and then close with a discussion of the verification environment.

Patterns from a file Another way for testbenches to get the input stimulus/control they need is to read the information from an operating system file. The file is generally formatted so that each record in the file represents either a specific input pattern to be applied at some time or a command that the testbench can interpret and in turn construct the input pattern to be applied. One benefit of this method is that only one simple generic testbench need be written. Changing test cases is as simple as telling the testbench to read a different test case file. In VHDL that means telling the simulator to simulate a different configuration. In Verilog that means supplying a new name for the pattern file parameter.

In the following example the first four columns in the file represent the values of four input ports. Usually, a pattern file also includes expected output for each input pattern. The last two columns are the expected values on two output ports. Thus, the pattern file includes all input values and all expected output values in every row.

A pattern file
0000 01
1000 11
1010 1Z
1001 00

Rows generally represent the passage of time. The time at which these patterns/commands are applied may be implicit or explicit. For example, an implicit time pattern may be applied every 10ns, or on rising edge of the clock (where clock is generated separately). An explicit time pattern may include offset times (or absolute times) in the pattern.

The testbench coder must insert code in the testbench so that it can read the appropriate files and understand the file data format. Verilog HDL contains the $readmemb or $readmemh system tasks to do the file read if the file data is formatted in a specific way using either binary or hexadecimal data. A user has some limited control of the data format. Another drawback of these routines is that they read the entire content of the file at the moment they are called during the simulation and the Verilog HDL simulation model must contain a memory structure large enough to hold all the pattern data. This memory structure could consume a lot of computer memory space, possibly causing paging and swapping during the simulation run. For more efficient computer memory utilization, the testbench memory could be loaded from several different smaller files at different times during the simulation.

Some Verilog HDL simulator tools offer custom utility functions to avoid the problems mentioned above, albeit at the expense of model portability. An alternate method in Verilog-XL is to use the $getpattern system task. This routine provides an efficient file handling and storage mechanism for reading patterns.

VHDL doesn't contain any built-in routines similar to $readmemb or $readmemh. Although the VHDL code necessary to duplicate $readmem or $getpattern is not hard to create, more than one person has already written this code. You would be well advised to do a little searching around to see what's available before you take the time to write yet another VHDL pattern file read program.

Pattern files are frequently used when test pattern data can be easily generated by some external method. For example, a graphical package may be used to draw input patterns in the form of 1's and 0's, the patterns could be read in by a testbench and applied to the design. Alternatively, nodes (signals, variables, registers, ports) in a design may be traced during one simulation of the design and then used as stimuli for later regression testing of the same design. In a similar manner the internal nodes of a larger or more abstract design can be captured and applied to the ports of some smaller, more detailed fragment of the same design. For instance, capture the stimulus and response of one partition of an RTL design and apply them to the synthesized gate level implementation of that partition.

Since binary or hexadecimal formatted pattern files are generally fairly large and since they take a significant piece of simulation run time to read, a good alternative is to place only the control/instruction information in the file. This will minimize the size of the external file, reduce the simulation time spent reading it and reduce the amount of space necessary to hold the file's data. The testbench now generates the actual detailed DUT inputs and expected results internally based on the information in the file. While this technique requires a smarter testbench, the efforts are more than paid back in terms of commensurate improvements in flexibility, reusability, and productivity. In the following example the first token in the file tells the testbench what operation should be applied to the DUT. The testbench interprets the control token and figures out what patterns, in what sequence, should be applied.

An instruction file
read 8 6
write ab94
int

There are no system tasks available in Verilog HDL to simplify reading a file like this. Rather, the testbench coder would have to write a C language routine using the PLI standard interface convention and call the C language routine from inside the Verilog HDL testbench. Likewise VHDL has no built-in routine available for this task but some tweaks to the VHDL routines mentioned above for reading patterns would rapidly result in a routine capable of reading this file.

Algorithmic operations It is natural to think of testing a DUT in terms of the functions it is expected to perform. Hence, tests can and should be developed as sequences of operations, where each operation may in turn be broken down into smaller operations. For example, a memory unit may be tested for write operations followed by read operations to the same locations to verify that the write operations succeeded. However, each write and read operation is in turn composed of the protocol sequence necessary to access the memory unit. Higher level operations such as cache-fill or DMA accesses may run for tens or hundreds of read or write operations.

These test operations are usually represented as either Verilog functions and tasks or VHDL functions and procedures. Alternatively VHDL guarded block descriptions or process descriptions can be used but parameter passing would have to be implicit. More complex test cases are created by calling these basic operations with appropriate arguments. For example, a write procedure may have address and data as two arguments. Calling this procedure once will test a simple memory operation.

Verilog code
write(200, 2)
VHDL code
write(addr=>200, data=>2)

Doing a DMA write access to 200 addresses requires inserting the procedure call into a larger procedure containing a loop statement. Of course, while this DMA is being performed, other concurrent operations may be testing other functionality in the DUT.

Algorithmic testing also points out a more sophisticated method of evaluating DUT expected responses. The partial and final results of a sequence of operations can be predicted or calculated. But rather than evaluate partial results for each individual operation in the sequence, it is only necessary to test the end result of the sequence. Intermediate results are only of interest if the end result is wrong. A variation on this is to have the DUT test itself under control of the testbench in the same way as BIST or signature analysis is used to detect manufacturing defects.

Figure 1. Each "cloud" represents a potential component of a test bench.

Simulated system environment Test-benches may be conveniently thought of as models that control the operation of the DUT. In reality the control operation may be only one piece of the testbench as shown in Figure 1. Large pieces of the testbench may be models of other portions of the sytem into which the DUT is incorporated.

In large designs that have several logical partitions, the designer could select the detailed level of some DUT partitions and select the higher, more abstract level models of the rest of the DUT. The higher level blocks should generate less simulation activity and so speed up the verification process of the DUT. The designer can also selectively enable or disable portions of the design that communicate with the DUT to force certain modes or operations, or simply to eliminate certain alternatives so they don't have to be considered.

Other models that might be included in the simulated system environment are:

• standard parts models,

  software-based models of standard hardware parts, obtained from some third party;

• full-function component models,

  special purpose, full-function software-based models,

  obtained from some third-party, or written locally;

• bus-functional models,

  special purpose, protocol only, partial function software based models of bus interfaces,

  obtained from some third-party, or written locally;

• real hardware components linked in through hardware modelers; or
• software models executing in hardware accelerators or emulators.

These various modeling techniques may be used to describe most anything from a collection of gates to a disk controller to a PCI bus interface. Note that each requires the existence of some particular interface to the simulator being used to simulate the testbench. The common thread is that each model contributes in some way to verifying the proper operation of the DUT.

In Verilog, the PLI interface communicates with actual hardware or external models and is used to drive inputs as well as compare output results. For example, a device expected to interact with a PCI bus may be interfaced to a real PCI bus in a real system through PLI.

Another testbench coding technique is to use the Verilog PLI interface or the VHDL foreign language interface to create C programs that drive or evaluate the DUT. This technique has been more popular in the Verilog world than in the VHDL world.

The Verilog PLI interface allows a C program to drive some or all of the Verilog HDL input ports through a user-defined C function attached to the PLI interface of a Verilog simulator.

Verilog code
always @(posedge clock)
$apply_pattern("vector.file",in1, in2,...inN);

We hope this discussion of design verification techniques has been helpful. As you can see there really is no "right way" to do simulation. There are only choices, each of which has its own set of advantages and drawbacks. Whether you choose to use randomly generated test patterns, discussed in the first part of this article and published last month, or whether you use file-based test pattern or algorithmically based test pattern generation techniques, you still need to consider the benefits of self-checking testbenches. And you also need to consider the components of the verification environment in light of your individual needs.

Larry Saunders and Yatin Trivedi are principals and cofounders of Seva Technologies Inc., a Fremont, CA-based technology consulting company.

To voice an opinion on this or any Integrated System Design article, please e-mail your message to: michael@asic.com.