In a large scale design verification project, change is the only certainty. It is a daunting task for management to look for objective measure of progress and estimate status and quality of the deliverables. Clear, unambiguous, real-time information is priceless for making informed decisions about the status and direction of the project. However, such information is buried deep in volumes of regression runs, is distributed, and is time consuming to extract.
Being agile in verification implies continued evolution of the verification strategy to deliver on the stated verification goals. Agility can only be achieved when decisions can be made based on all pertinent, real-time and historic information about the project.
A system providing comprehensive information that gives the users a clear indication of the progress of verification projects is an indispensable tool. Such a system must enable engineering teams to collaborate, while at the same time present a source of real-time, up-to-the-minute, information for the management. In addition, heuristics based analysis of historic data could be valuable in making informed decisions about the projects.
Agile verification ensures that resources will not be squandered away, meaningful deadlines will be established, and high quality of results will be delivered.
Design verification is estimated to consume about 70 percent of resources for chip development. A clear understanding of how so much resource is being spent often eludes management and engineering teams alike. With rising complexity of designs, verification teams are on the lookout for effective methodologies that can reduce the risk of failure, streamline verification activity and provide management the required systematic information about progress and quality.
At the center of all this, is the development of a verification plan, which is often a laborious ritual held at the start of a large project only to be cast aside as the project unfolds. Developing a verification plan is one thing and keeping it in sync with the requirements, and changing verification strategy, is quite another. Attempts to manually keep the requirements, the plan, and the execution environment in-sync is time consuming and unreliable to say the least. Analysis of execution results is a daunting task due to the sheer volume to the data generated in the process. Often, distributed teams executing the plan add another dimension of the complexity.
Agile methods for software development have proven themselves over the years since their first introduction in 2001. Such projects are characterized by higher success rates as evidenced by higher productivity, better quality, lower costs, and overall higher satisfaction.
In order to achieve such success rates in design verification projects, what is needed is a methodology and supporting tools that uses agile principles to keep the verification plan, requirements, and all pertinent data in a central, dynamic, accesses controlled system from which the verification teams and management can come up with a strategy for success.
The verification management problem
Let’s look at the problems that we are trying to address in some detail. These problems plague many a modern verification teams.