The verification project was split between the two companies working together across five sites on two continents. Project managers were challenged to meet the high quality requirements of the automotive safety industry, so we needed to organize the verification teams to achieve maximum efficiency.

Our documentation needs would be highly demanding as we worked toward certification for the safety standard IEC 61508. On previous projects, we found the design documentation provided a starting point for our verification engineers to extract the design features and write a verification plan. However, this was far from efficient because the approach offered no clear linkage and the process was entirely manual.

Also on previous projects, we used a process where all tests were implemented and added to the regression list so they could be rerun with each new revision of the design database. The goal was clear: get all the tests to pass. However, there was no direct form of feedback to substantiate whether or not the verification plan was fulfilled. While we wanted better linkage between the regression results, verification plan, and the design documents, we lacked the ability to automate this process. In order for us to get a framework for the verification work across the various work groups, our flow had to be manually pre-planned and documented.

Click on image to enlarge.

The verification plan or "vPlan" is also written in FrameMaker, using the verification plan formatting rules. Enterprise Manager has the ability to read the vPlan document (when saved in xml or PDF format). It also controls the regression start and generates results reports. With the vPlan, it is also possible to map the results back to the verification plan. The mapping step provides a clear picture of the status of the verification work relative to the overall plan. This is a vital feature because it enables management to closely track the progress of the verification tasks on a feature by feature basis, or even down to the technical details of each feature. The vPlan and its results are hierarchical, so viewing becomes organized and very efficient.

Click on image to enlarge.

In this project, some IP blocks were integrated several times. The "Verification Scope" parameter defines which instance of IP blocks for which the table is valid. Enterprise Manager VSIF files (which define the tests to run in the regression) need to have the same definition inside.

vPlan References provide another means to reduce the complexity of the plan. Standard interfaces to be verified for a number of IP blocks have separate verification plan tables, which are simply referenced later. The approach unifies the verification of the interfaces and shortens the write-up of the tables for the IP blocks.

The regression setup for Incisive Enterprise Manager was as follows:

. Central script setup and central attribute definitions for JDP project ensures exchangeability of VSIF files between SoC projects.

. Execution of test cases in 3 phases, handled by scripts:
o Compilation of database in pre-session (single compile step to create all necessary executables).
o Simulation of test cases in parallel via LSF. Run script handles all arguments defined by attributes.
o Reporting step in post-session (optional).

. Configuration of DRM setup (e.g. LSF options) done by each site in a predefined way and included via $ENV variables in project setup scripts.

Click on image to enlarge.

Here's an example of a vPlanTree report:

Click on image to enlarge.

The report is very useful for informing the project team and management frequently. Progress is accurately reported and structured by IP and resources. The report reveals the percentage of implemented, or respectively, passing tests, which help the team understand the risks and direct subsequent design and verification efforts. Incisive Formal Verifier (IFV) integration into Enterprise Manager
As with any modern complex design, part of the verification must be performed through property checking. In our case, beside specific block-level verification, we used IFV for connectivity (and multiplexing) top-level checks. Then, for regression tests, we encountered a problem merging results from two very different worlds: simulation and property checking.

However, we were using version 8.2 of the Cadence' verification suite which had just introduced a "unified coverage model" for collection of various coverage types.

For the 8.2 release, the integration went through the following steps:

. Extend central attributes to handle specific IFV parameters (e.g. engine, effort, halo, etc.);

. Extend the pre-session to create the IFV suitable snapshot (macro redefinition, blackboxing, "formalbuild" step);

. Add specific scripts to handle IFV run with proper usage of additional attributes (written adapting some example scripts provided by Cadence);

.The referencing in the vPlan is done in the same way as for simulation jobs (each assertion is treated as a single test).

As a result, IFV can be run from Enterprise Manager, together with all other regression tests. This delivers a big advantage because Enterprise Manager can be used as the central place to run the whole regression and analyze all results.

In addition, we have found many improvements with the introduction of version 9.2 of the Cadence verification suite. Now, Enterprise Manager installation comes with pre-defined IFV attributes (hence eliminating step 1), together with a pre-defined IFV run script that can be easily referenced inside the VSIF (hence eliminating step 3). As a result, integrating IFV regression inside any existing Enterprise Manager environment is now much easier. You simply add the compilation phase. Tool issues and enhancement requests
Our project utilized version 8.2, however, we have recently shifted to the latest 9.2 release (October 2009). We've found a few areas where we would like to see some revisions made to improve the process.

. More customized reporting would be helpful because we'd rather not task the different project teams with post-processing needs. Comparing the results with previous regression runs is possible, but the format could allow more visibility.

. We didn't use Enterprise Planner, so it wasn't possible to do a pre-check between the VSIF files and the FrameMaker-based vPlan. This resulted in cumbersome iterations. Similarly, only Enterprise Planner would allow a syntax check of the VSIF files before starting the regression.

. For everyday users who are not knowledgeable of the VSIF file syntax, the possibility to set parameters in the GUI directly would be a nice advantage. Projects with large numbers of tests and parameters could even use a scripting language in the VSIF files.

. When using IFV, we have to carefully check assertions are not failing and are fully covered. This has to be assured by the user in the planning phase.

. Using Enterprise Planner would give us an advantage. However, if we move in this direction, we wouldn't be able to continue using FrameMaker to write and maintain the plan. So, we have been cautious to take this step. We fear specification tagging and automatic generation of vPlan tables would have to be supported in a corresponding way in Enterprise Planner. We are working currently with Cadence on this because we believe the feature set is very promising.

Conclusion
Our new workflow with Cadence's Incisive Enterprise Manager contributed significantly to a more organized and efficient verification project. Mapping the results directly back to the vPlan provided a very clear reporting model and helped us refine our plan to accelerate closure. Though we lost some time in maintaining the references to the test definitions in the plan (link between vPlan and VSIF files), we are working with Cadence to better understand how future improvements like Enterprise Planner will solve this problem for our projects going forward.

About the authors:
. Frank Donner works on verification in Freescale's New Product Development Center in Munich, Germany. He is currently doing testbench integration and verification on a 32-bit Power PC-based SoC.

. Joachim Fader works on design and hardware verification in Freescale's New Product Development Center in Munich, Germany. Currently he works as a verification lead for products for the automotive industry based on the 32-bit Power Architecture.

. Michael Pallas works on verification in Freescale's New Product Development Center in Munich, Germany. Currently, he works as a verification engineer responsible for regression flow and functional verification on a 32-bit Power PC-based SoC.

. Umberto Rossi works on verification in STMicroelectronics' Automotive System-to-Silicon group in Agrate, Italy. Areas of interests include functional verification, formal verification and validation on 32-bit Power PC-based SoCs.

. Franco Toto works on verification in STMicroelectronics's Automotive Product Group in Agrate, Italy. Areas of interest include model checking, hybrid techniques (static/dynamic verification), and functional verification.