Design Article
Managing coverage grading in complex multicore microprocessor environments
James Young and Michael Sanders, AMD Inc., Paul Graykowski and Vernon Lee, Synopsys Inc.
1/19/2011 9:19 AM EST
Quickgrade architecture
Quickgrade is a UCAPI based program which provides a methodology for incremental test grading on code coverage metrics. The algorithm supports all code coverage metrics, and is easily expandable to assertion and functional coverage. As utilized by AMD, the focus is on line and toggle coverage metrics. Quickgrade accepts a test list and coverage database as inputs:
The tests are graded in the order specified in the test list, listOfTestsToGrade. Each test is individually loaded, then its coverage metrics are analyzed and compared against the total of the previously loaded tests. Results are then presented in a tabular form that highlights the contribution for each test per coverage metric. A running tally of total coverage is also kept.
The main reason why Quickgrade is so fast is that it computes each test’s incremental coverage only once, in the order specified in the file listOfTestsToGrade. While it does not produce a fully optimized list, it does provide a high-quality ordered list in increasing level of contribution which can be used to extract an efficient and effective regression list.
The first step is to load one or more design directories, which contain the simulation collected coverage data, into the database. Systematically the listOfTestsToGrade is extracted one by one, starting by loading the first test and extracting the coverage information. Each additional test is then loaded and merged into the preceding such that a running total of progress is maintained.
Figure 1 – Quickgrade flow diagram
For each loaded and merged test, all the source instances (module instances) are progressively scanned for the total number of coverable objects and the total number of covered objects. A running tally is maintained for each coverage metric.
In the previous code, ‘covered’ is the sum of all covered objects for all instances. Note the UCAPI property covdbDeepCovered returns the total coverable objects for each design module instance. The value allcoverable[m] stores the total number of coverable objects for each metric using the UCAPI property covdbDeepCoverable at the instance level. The property covdbDeepCovered is called at the test level to get the covered items for each test, while covdbDeepCoverable is called at the design level to get the total coverable for the design, i.e., we want to be able to look at total possible coverable objects for a given design and then look at the contribution of each test to the number of covered items.
Once the number of covered and coverable objects is obtained, the incremental value is calculated:
The score is then calculated for each test:
Similarly, the total score is reported as:
Sample output from a small run is illustrated below:
Next: Applying Quickgrade
Quickgrade is a UCAPI based program which provides a methodology for incremental test grading on code coverage metrics. The algorithm supports all code coverage metrics, and is easily expandable to assertion and functional coverage. As utilized by AMD, the focus is on line and toggle coverage metrics. Quickgrade accepts a test list and coverage database as inputs:
./quickgrade listOfTestsToGrade simv.cm
The tests are graded in the order specified in the test list, listOfTestsToGrade. Each test is individually loaded, then its coverage metrics are analyzed and compared against the total of the previously loaded tests. Results are then presented in a tabular form that highlights the contribution for each test per coverage metric. A running tally of total coverage is also kept.
The main reason why Quickgrade is so fast is that it computes each test’s incremental coverage only once, in the order specified in the file listOfTestsToGrade. While it does not produce a fully optimized list, it does provide a high-quality ordered list in increasing level of contribution which can be used to extract an efficient and effective regression list.
The first step is to load one or more design directories, which contain the simulation collected coverage data, into the database. Systematically the listOfTestsToGrade is extracted one by one, starting by loading the first test and extracting the coverage information. Each additional test is then loaded and merged into the preceding such that a running total of progress is maintained.
Figure 1 – Quickgrade flow diagram
For each loaded and merged test, all the source instances (module instances) are progressively scanned for the total number of coverable objects and the total number of covered objects. A running tally is maintained for each coverage metric.
In the previous code, ‘covered’ is the sum of all covered objects for all instances. Note the UCAPI property covdbDeepCovered returns the total coverable objects for each design module instance. The value allcoverable[m] stores the total number of coverable objects for each metric using the UCAPI property covdbDeepCoverable at the instance level. The property covdbDeepCovered is called at the test level to get the covered items for each test, while covdbDeepCoverable is called at the design level to get the total coverable for the design, i.e., we want to be able to look at total possible coverable objects for a given design and then look at the contribution of each test to the number of covered items.
Once the number of covered and coverable objects is obtained, the incremental value is calculated:
incr = covered – allcovered[m];
The score is then calculated for each test:
metincrscore = (100.0 * incr) / allcoverable[m];
The score, metincrscore, is a score for each test which represents the incremental number of covered items relative to the total number of possible coverable items.Similarly, the total score is reported as:
metscore = (100.0 * covered) / allcoverable[m] ;
Sample output from a small run is illustrated below:
Next: Applying Quickgrade
|
|
|
|
|
Navigate to related information