Analyzing code coverage
In code coverage analysis, information
about the RTL code executed is recorded during a regression by the
simulator. This is a fully automated process. However, after the
regression has finished and the code coverage from all tests run has
been merged into one database, the result needs to be analyzed, which is
usually done in a review by the design or verification engineer. They
need to look at the parts of the code that were not completely
stimulated during the regression.
Fortunately not all of these so-called “code coverage holes” need to
be manually reviewed. Specifically, there are two types of code
coverage holes: ones that could actually have been stimulated and ones
that are impossible to stimulate. The latter kind comes, for example,
from reuse of submodules in a hierarchically higher context where some
inputs may be tied or not all features become accessible. It could
significantly reduce the effort spent on the reviews if these could
somehow be sorted out.
Figure 2 – Not all coverage holes require detailed manual review
there is a way: Formal analysis tools can create a condition for every
block of code or expression in the code that needs to be met in order
for that part of the code to become stimulated. The formal tools can
then check whether this condition can actually become true. In Cadence®
Incisive® Enterprise Verifier, this is now fully automated.
a regression has finished, the code coverage from all runs first needs
to be merged into one database. This database is then provided to
Incisive Enterprise Verifier as an input. It will look for all code
coverage holes in this database and then run an automatically generated
property to check whether the hole is actually reachable or not. If it
determines this hole to be unreachable, it will add this to a list of
code coverage hole exceptions and will at the end write these out into
an exception file. This can then be applied to the merged code coverage
database and a new report with code coverage holes can be written
out—but without all unreachable holes.
All of this can be done
in a post-session script of the Incisive Enterprise Manager tool from
Cadence (which is used to run regressions called “sessions”) in a fully
automated manner. So whenever a regression finishes, the resulting code
coverage databases are merged, analyzed for holes, and an
already-filtered code coverage hole report is written out for the
When the above flow is applied, several things will come to the forefront.
example, if the design has a test mode, it will most likely have
something like test multiplexers controlled by an external test enable
port. Another common case is where the design has an external interface
using a protocol that contains some constraints for the design’s inputs.
In cases like these, you would expect dead code caused by these
external constraints to be marked as unreachable in the code coverage
However, as noted above, the flow is (so far)
fully automatic and no design-specific setup is done. So there are no
constraints that would model any external influence on the module (of a
tied port or a protocol).
Now, it would be easy to add these to the code coverage hole
analysis; however, we decided against doing so because we are performing
code coverage analysis for the single purpose of finding errors in the
verification execution. If the verification engineer had misinterpreted
an input signal to the design and falsely tied it, or if he/she
misinterpreted a protocol and implemented it incorrectly, we would like
these to show up in either the functional coverage (which is collected
according to the verification engineer’s coverage collection
implementation and might also be erroneous) or in the code coverage
In an exhibition that I recently attended, where Fujitsu is one of the exhibitors, they covered this analysis extensively in one of the breakout sessions they had for those interested. It was quite informative, but at the same time, too information heavy that many of the audience left more baffled rather than clarified. I would suggest that Fujitsu take extra care to have high level of information provided during exhibitions, and hopefully bring the discussion at a different time and venue for those who would like to take it further. The article seems to have covered most of it, and I am glad to have stumbled upon this after their presentation. http://www.discountdisplays.co.uk
Join our online Radio Show on Friday 11th July starting at 2:00pm Eastern, when EETimes editor of all things fun and interesting, Max Maxfield, and embedded systems expert, Jack Ganssle, will debate as to just what is, and is not, and embedded system.