Coverage Doesn't Have To Be A Dead-End Street

Code coverage tools have made considerable advances in a few short years, but the third-generation tools must address the thorny issues of functional verification and linkage to test generation.

by Michael (Mac) T.Y. McNamara

Coverage analysis software burst onto the verification scene in recent years and soon captured market share by its ability to measure the design code exercised by a test suite. Prior to the advent of code coverage tools, you "knew" you were done with RTL simulation and verification when you hadn't found a new bug in the last week (or two weeks, or two days, or . . .), or when it was time to tape out and the engineering manager said to stop. Code coverage has benefited the design and verification team by giving one objective measure of verification completeness.

Code coverage still raises two issues. First, code coverage tools provide metrics that don't directly address the functional verification of the design. Second, code coverage is a bit of a dead-end street. Think about it--what do you do after you measure coverage?

Code coverage tools provide information important to the measurement of functional coverage. You know that if you haven't tested the logic, it doesn't work. But testing individual design elements doesn't guarantee correct design functionality. Knowing that you've exercised all the blocks in a case statement doesn't ensure that your state machine works correctly. Vendors have made progress; some second-generation coverage tools let the user query the coverage database interactively to determine if the tests have exercised specific paths. This feature allows the user to check more than just lines or branches or states. It's at least a start.

These second-generation tools also pave the way for more widespread usage of coverage techniques by reducing run times by as much as 10 times and by automatically recognizing state machines in the design, making engineers and simulation licenses more productive.

So what do you do after measuring coverage? Typically, you start a long and painful loop of writing more tests, running simulation, and measuring coverage--only to find out that your new tests haven't substantially increased the amount of the design tested. There's that brick wall at the end of the dead-end street, and you're pounding your head against it.

Therefore, the goal of the next-generation verification tool suite should be to automate, reduce, and even eliminate this loop. The coverage tool already knows what parts of the design haven't yet been exercised. So extending the coverage tool, and linking it to test generation, would turn coverage from a dead-end street into the verification highway.

The test generation tool should perform two primary tasks. First, it should automatically generate the additional test vectors that increase coverage from the value obtained with the user-developed test suite to close to 100 percent. Second, it should run in a semiautomated mode, where you can pick and choose the (as yet untested) elements of the design that the test generation tool should target.

Additional requirements should fall on the test generation tool. The test vectors it generates should be in the design source language and include full comments. You shouldn't have to learn a completely new language to run the tool. The generator should drive a reference model and compare simulation results from the design and from the model. In addition, it should allow you to constrain the test vectors it generates, to ensure that it creates only legal test vectors.

A coverage-based test generation tool might also generate pseudo-random test vectors to exercise the complete design. A common verification scenario uses random vectors to more thoroughly exercise the design in regression tests. A coverage-based test generation tool would generate vectors that target specific design elements. Instead of exercising part of the design 100 times and another part perhaps not at all, this tool would yield a more complete set of vectors.

Other usage models and applications are easy to imagine. In semiautomated mode, for example, designers could use the test generation tool early in the design cycle to generate test vectors at the module or functional block level. Whatever the application, the message remains the same: Linking code coverage and test generation means that coverage analysis doesn't have to be a dead-end street anymore.

Michael (Mac) McNamara is president and cofounder of Surefire Verification (Campbell, Calif.), formerly known as Silicon Sorcery. He was on the team that designed the VCS Logic Simulator at Chronologic Simulation, and served as the company's vice president of engineering from 1992 to 1995.

To voice an opinion on this or any other article in Integrated System Design, please e-mail your comments to jeff@isdmag.com.