Well, it's happening again, but this time we're in Functional Test Space (FSM) and it's David Murray, an Irish engineer, who is flailing his arms, warning us of an impend ing problem.

"I have seen numerous methodologies which use coverage tools to pick out regression tests that produce the highest code coverage. This is where a perception flaw can happen," warns Murray. "A very simple example of this follows. Take state 'B' to be the wait state in an arbitration sequence of a state machine, like so: A->B, B->B OR C, C->. . . . We see that B can transition to B or C. This is its logical transition."

"However, the following functional scenarios can happen: A->B->C (a fast arbitration), A->B->B->C (a normal arbitration) or A->B->B->B->C (a slow arbitration)," continues Murray. "Now, let's have two tests. Test One has 95 percent code coverage but only covers one of the arbitration sequences. Test Two has 70 percent code coverage but covers all of the sequences.

"According to many code-coverage methodologies, we should pick Test One. It has the highest code coverage. However, what about Test Two?

" This would have more functional coverage than Test One," adds Murray. "Isn't this what we want at the end of the day?

"I think that a blend of the two is what is needed. We just need to be aware of the weakness of code coverage to use it to its full potential."

Murray goes on to warn, "Code coverage is linked to functional coverage. The link can be weak or strong, depending on the type of design, but code coverage rarely equals functional coverage.

"Introducing a code-coverage tool without proper planning can sometimes be a distraction to an inexperienced designer. If a code-coverage tool is used iteratively as designers write tests to verify their block, then they can very easily find themselves in a feedback process whose focus tends toward optimal code coverage rather than optimal functional coverage.

"So they can be confident that their code is fully covered but wrongly confident that the functionality is covered."

The real fix for the problem would be next-generation code-coverage tools that are aware of how time-based FSM games affect functional testing. None really do this now, but I'll be keeping my eyes open for them at DAC '99.

"In my opinion, code-coverage tools shouldn't be trusted, but they should definitely be used. Though it sounds like a contradiction, I think that this is the best way to use them," concludes Murray. "A certain amount of distrust is healthy."

-John Cooley runs the E-Mail Synopsys Users Group (ESNUG), is a contract ASIC designer and loves hearing from engineers at jcooley@world.std.com or (508) 429-4357.