Inside HDL Testbench Tutorial, Part I

What is the right way to verify models? The Inside HDL column this month is part one of Larry Saunders and Yatin Trivedi's testbench development tutorial.

By Larry Saunders and Yatin Trivedi

We are often asked to explain the "right way" to verify the correctness of a model of some particular piece of hardware. Since it is almost never the case that there is only one "right way" to verify a model's correct operation, and since there are many verification strategies to choose from, we thought we would provide a little background about design verification and then review some of the approaches used by designers for validating their simulation models.

In the first part of this two-part column we will provide some background color, discuss self-checking testbenches, and talk about the uses of randomly generated test patterns. In part two next month, we will continue our review of verification approaches covering the uses of file-based test pattern and algorithmically-based test pattern generation techniques, and close with a discussion of the verification environment.

First, though, a comment about that word "testbench."

More than one engineer has wondered where this silly word came from and why in the world anyone would use it to describe simulation control models. While the real origin of the term is lost, at least to us, the first we remember hearing it was back in the early days of VHDL when it was used by the original VHDL development team to refer to VHDL models that controlled the simulation of other VHDL models. To explain the word (and cement the analogy in the process) there was even a picture in a manual showing a work bench with waveform generator boxes driving the inputs to a "VHDL model" and oscilloscopes attached to the outputs.

Some may argue that in spite of the obvious analogy, it's still a silly word to use. We think the truth of the matter is that the word "testbench" has caught on for one simple reason: the lack of a better word. While "testbench" doesn't have any sex appeal, it also doesn't have any competition. Which slips off your tongue more easily: "simulation control model" or "testbench?" What other choices are there? And just to mark its acceptance into EDA jargon, let us point out that the word testbench has now broken the bonds of its VHDL heritage and has quietly slipped into the idiom of the Verilog world.

What's a testbench? In order to test the model of some design, a designer must apply test patterns to the input ports and observe the output ports over time to decide whether the inputs were transformed to the expected outputs. The model is generally referred to as the design under test (DUT). By testing, we mean verifying functional correctness, not finding manufacturing faults. The most common method used to accomplish this is to physically surround the DUT with a layer of code that performs the stimulus generation and output comparison (see Figure 1). This layer of code, known as a testbench, can be very simple or as complex as necessary for the application at hand.

Figure 1. A testbench consists of code to generate patterns to apply to the DUT and other code to monitor the results.

Although testbench code has traditionally been coded using a simulator-specific simulation control language, we are going to recommend highly that the testbench be written using the same language as the DUT, whether VHDL or Verilog. The reason is simple: model transportability. These days, very few design models can be expected to spend their entire existence running on the same simulator. Sooner or later they are going to be sent across the street or across the country to some other group that uses a different simulator. If the testbench can't make the trip along with the DUT, a critical analysis capability has been totally lost.

Generally, the testbench is created as a new hierarchical level by defining a VHDL entity/architecture pair or a Verilog HDL module with no ports, one or more instances of the DUT and arbitrarily structured pieces of code to generate and apply test patterns and observe results. All of these code fragments are interconnected via signals. This new hierarchical level is frequently named (or referred to as) system, top, test, testbench, or environment.

The term "testbench" is somewhat generic in that it generally includes all the stimulus generation and output comparison applied to some DUT. Some unique set of input stimuli created to test a unique operation of the DUT is referred to as a test case, and one input vector from the test case is usually called a test pattern. A reference to a DUT's testbench is generally considered a reference to all the test patterns in all the test cases associated with that DUT.

Listing 1 
Generating random vectors
 
Verilog HDL code
always @(negedge clock) 
databus = $random; 
VHDL code 
process (clock) begin 
if clock='0' then random(databus,seed);

In general, testbenches have two goals: to stimulate the input ports of a design and to verify correct outputs. The inputs applied may be periodic (e.g. every 20ns), synchronous with some reference signal such as a clock, or even completely random. Some testbenches apply patterns by taking into account feedback from past outputs (maybe the last output value, but sometimes from several cycles back). In such cases, the output is sampled, and a decision is made as to what pattern will be applied next.

Self-checking testbenches Most "intelligent" testbenches also provide a validation mechanism for the vectors applied. These are called self-checking testbenches. This means the testbench does some form of output sampling of the design under test and compares the sampled values to expected results. If actual outputs match expected outputs, fine. If there are differences, the testbench provides information about what was being tested, the inputs applied, the actual outputs detected, and expected results. Obviously the self-checking portion of the testbench doesn't spring full blown from the testbench coder's mind. Rather, it is an iterative part of the testbench creation and debug process: write some input patterns along with the expected results checking code, simulate the test case to identify problems in the self-checking code, fix the code, and repeat.

Listing 2 
Using random number generators to cause variable delays between two
operations

Verilog HDL code
always @ (posedge clock) // sync with clock edge 
...
begin
# (($random % 10)*clock_cycles) // delay between 0 and 10 clock cycles
apply_op1(...); // do a test
# (($random % 10)*clock_cycles) // delay between 0 and 10 clock cycles
apply_op2(...); // do another test
VHDL code
process 
...
begin
wait until clock='1'; -- sync with clock edge
random(cycles,seed); -- calculate cycles to
delay
wait for cycles*clock_period; -- wait
apply_op1(...); -- do a test
random(cycles,seed); -- calculate cycles to delay
wait for cycles*clock_period; -- wait
apply_op2(...); -- do another test

Sometimes in the rush to finish a project, the self-checking output verification portion of a testbench is left undone. Very often this is a decision much regretted later. Although the output checking code takes extra time to insert in the testbench, the productivity reward is large. The mere act of writing the output checking code forces the

Listing 3 
Generating random ALU operations


Verilog HDL code
always @(negedge clock) begin
alu_func = $random;
...
VHDL code
process (clock) begin
if clock='0' then random(alu_func,seed);
...

Listing 4 
Using special statistical distribution functions


Verilog HDL code
data1 = $dist_uniform(...);
VHDL code
dist_uniform(data1,seed);

coder to think about what the results should be and when they will be available to look at. Just thinking through the functional operation to the expected results is valuable information during DUT debug (and testbench debug), as it may offer insights about potential design problems or additional test cases that need be incorporated into the testbench.

Since the life of a testbench may be eternal, the output checking code also functions as embedded documentation of the intent of the testbench. Trying to understand the operation of an unfamiliar testbench written by another engineer, or trying to relearn the operation of a long forgotten testbench you wrote will quickly demonstrate the value of the self checking code.

If a self-checking testbench can provide documentation of intent to a curious reader, then it is also true that a self-checking testbench never forgets what needs to be checked, it never gets tired or careless, and it never changes its mind about what's important. Testbenches generally end up in some regression test library being used for "go/no go" screening after they are debugged. It goes without saying that testbenches that need to be manually screened for correct operation every time they're run are going to waste a lot of someone's time, and maybe--just maybe--someone will overlook something.

The issue of verifying simulation results also points out one of the significant differences between VHDL and Verilog. While testbenches in either language have no trouble observing the output ports of the DUT, observing internal points within the hierarchy of the DUT can be a problem in VHDL. Observing an internal point is considered intrusive because it accesses the design state without going through the port interface structures.

Verilog allows the use of path names to let code in one declarative region reach outside of its scope and read the value of an object in another declarative region. While this action would be considered really bad form in a hardware model--the equivalent of soldering a wire to the middle of an IC die--it makes perfect sense in a testbench model. After all there is no intent to make hardware out of the testbench. VHDL does not allow this path name referencing because it assumes that all declarative regions are part of some hardware model.

Fortunately there is an adequate, if not elegant, solution within VHDL: global signals. Declare some signals in a package, make the

Listing 5 
Comparing pre-synthesis RTL models with post-synthesis gate-level models


Verilog HDL code
uut1 uu1 (...,out1); // include the two alternative modules
uut2 uu2 (...,out2);
always @ (posedge clk) begin // positive clk edge
# 20 // wait 20 ns
if (out1 != out2)
$display ("outputs are different at ", $time);
// compare outputs
end
VHDL code
uut1: uut port map (...,out1); -- alternative architectures
uut2: uut port map (...,out2); -- selected in configuration
assert not -- assert wants false expression
((out1/=out2) and -- outputs not equal at
clk'delayed(20 ns)='1' and -- 20 ns from positive clock edge
not clk'delayed(20 ns)'stable)) -- (yes, it could be simplified)
report "outputs are different"
severity warning;

package visible to both the DUT declarative region of interest and the testbench, assign the value of interest within the DUT to the global signal, and reference the global signal in the testbench. It's kind of messy, but it works.

Testbench coding strategies Testbenches produce input patterns for the DUT in one of three ways: randomly, using some scheme to produce random data or events, from an external source such as reading patterns from a file, or algorithmically, using routines embedded in the testbench. Combinations of the three alternatives may also be useful. Any of these testbench coding schemes are equally applicable to testing high-level performance models, high-level behavioral models or register transfer level, synthesizeable models, and gate-level models.

Random vectors Several variations exist under this broad theme. One alternative is to create random data values with a random number generator. This technique is useful if the actual data being applied doesn't matter for testing: anything should do. It may be especially useful if many different data values need to be tried because the operation being tested is known or suspected to be sensitive to specific data combinations. In the example in Listing 1, the databus is assigned a random value.

Random number generators are also useful for generating volume varying streams of stimulus activity to be applied to the DUT. In the example in Listing 2, random number generators cause variable delays between two operations.

Another approach is to use a random number generator to select the next operation to be applied, the order of operations to be applied, or the number of operations to be applied to the input of the design. In Listing 3, the selection and order of ALU operations to be performed is decided using random.

Of course not all random situations conform to the bell curve; various statistical distribution functions may be used for generating test cases (see Listing 4).

Test cases using random selection of input control information do not easily lend themselves to automatic checking of outputs against expected results unless the expected outputs can be independently computed using the same inputs being applied to the DUT. If so, the independent calculation can provide the expected output value to be compared against the DUT computed value. For example, a testbench tasked with testing Booth's multiplication algorithm (shift and add) may apply two random numbers to the DUT and independently compute the expected results inside the testbench.

If multiple versions of the DUT exist, another alternative is to create a testbench that includes both versions, broadside the same input patterns to both versions, and compare the output results. A typical example (see Listing 5) is comparing a register transfer level, synthesizeable model with the post synthesis gate level model. Caution is in order here because the outputs may not be identical at every moment. It may be necessary to wait and sample the outputs only after they have settled down.

In part two next month we will continue our discussion of testbenches and verification approaches.

Larry Saunders and Yatin Trivedi are principals and cofounders of Seva Technologies Inc., a Fremont, CA-based technology consulting company.

To voice an opinion on this or any Integrated System Design article, please e-mail your message to: michael@asic.com.

integrated system design  April 1995