This is the third in a series of four articles outlining a reference verification methodology that covers both RTL and system-level requirements. This methodology is enabled by the SystemVerilog hardware design and verification language standard and is documented in the Verification Methodology Manual (VMM) for SystemVerilog, a book jointly authored by ARM and Synopsys. This article focuses on the techniques recommended by the VMM for SystemVerilog for ESL verification.

Introduction to system-level verification

In its most general terms, an electronic system is a design comprising independently designed and independently verified blocks as well as a block interconnect infrastructure. System-level verification is the verification of the correct interaction among these individual blocks.

Each block, including the block interconnect infrastructure, must be verified individually. Therefore, system-level verification should focus on the functionality embodied by the combination of the blocks. Any functionality that is entirely contained within a block is better verified at the block level.

System-level verification can take place on systems of any size, from tens of thousands to tens of millions of gates. In all cases, the principal requirement of verifying that the specified functionality and performance goals have been met is the same.

The system architect establishes performance, latency, and bandwidth goals for the various components of the system. The implementation teams must meet these goals. The verification teams, along with demonstrating that these goals are met, must verify all corner cases introduced by the implementation to demonstrate that an invalid state or behavior cannot be reached.

The biggest challenge facing a system design team is not in the specification and integration of many design blocks, but rather in achieving a confidence level in the correctness of the final design. This article describes a methodology and verification tasks applied to system-level verification. It should be of interest to block and system-level architects and verification engineers, with a focus on system-level integration verification.

Extensible verification components

The layered-testbench approach discussed in the second article in this series recommended the creation of individual transactors that can be reused from block-level to system-level environments. When considering these transactors from a system-level perspective, it may be necessary to extend or combine their block-level functionality into system-level functionality. Individual transactors can be structured into system-level transactors, each referred to as an extensible verification component (XVC).

XVCs provide a foundation for modular, scalable and reusable system-level verification environments, with the aim of minimizing test setup overhead. XVCs can be used to drive block interconnect infrastructures or external interfaces. They can also support other XVC components by monitoring system state and providing notification information.

The purpose of an XVC is to support system-level integration and functional verification using both directed and constrained-random testing under a unified methodology approach. The structure of an XVC is such that it is highly portable to different system-level designs and across design abstraction levels.

An XVC can encapsulate verification expertise and system-level functions in a standard way. Despite potentially having widely varying levels of functionality, the user of an XVC is always presented with the same look and feel at a testbench level, which contributes to a reduced learning curve and enables a consistent system-level testbench control mechanism.

An XVC is a container for verification IP divided into two layers, as shown in Figure 1. The generator layer is a user-extensible library of test actions that can be executed by the XVC, with a defined action interface for integration into a verification environment. The driver layer integrates individual transactors for implementing the actions on a physical-level or transaction-level interface interacting with the design-under-test (DUT). The generator layer controls the transactors in the driver layer.

Figure 1 — The XVC is structured into two layers: generator and driver.

The action interface of the generator layer allows coordination of test stimulus. The generator layer verification environment interface of an XVC can connect directly to an XVC manager. An XVC manager can support any number of XVCs at one time and uses this interface to schedule execution of individual actions within any given XVC. XVCs can also pass data and status back up to the test control infrastructure to communicate with other XVCs in the same environment. Experience shows that most of the system verification issues are observed when blocks compete for common system resources. To create the necessary competitive scenarios, it is not sufficient to simply execute a single-threaded test. Similarly, it is unlikely that independent stimulus streams on independent external interfaces will create the required scenarios.

Verification components must therefore be coordinated to create the required stimulus. This coordination is easiest to achieve when a single, central XVC manager coordinates the execution of multiple XVCs.

The XVC manager

The XVC manager is an optional verification component responsible for the high-level synchronization of XVCs. The synchronization and XVC control mechanisms can be user-defined according to the need of the system or a specific test. There should be only one instance of the XVC manager in a testbench environment; using multiple managers would require an additional level of coordination among them.

The VMM for SystemVerilog specifies a predefined XVC manager that uses test scenario descriptions written as external configuration files in a plain text format. These files can be reused or modified to new requirements with relative ease. The ability to direct the test via simple text input files enables the user to quickly achieve effective test results without a detailed understanding of the internals of the verification environment.

Having test scenarios specified as external files also eliminates the need for any re-compilation of testbench components or the design to run different test scenarios. This specification in turn helps to minimize test turnaround times for large and complex system designs. Figure 2 shows the structure of the predefined XVC manager.

The concept of a test scenario is fundamental to the function of the predefined XVC manager. The predefined XVC manager directs XVCs within the scope of a test scenario. A system-level test program can contain one or more test scenarios. Test scenarios are designed to meet one or more verification requirements.

A test program might consist of a number of test scenarios that, when taken together, fulfill several test requirements for a DUT. Test scenarios can also be used to encapsulate common sequences of actions so they can be reused by different test programs. For example, the operations required of a number of XVCs to configure a system could be contained in a scenario.

Test scenario files for the predefined XVC manager are fixed prior to run time and cannot be dynamically modified. A test program includes the descriptions of test scenarios and information regarding the order in which each scenario is executed. Scenarios may be repeated any number of times and in any order.

Every test action that can be executed in the test program is checked by its target XVC before executing the first action. This checking prevents a semantic or syntax error in the scenario file from aborting a simulation after a long run.

System-level verification environments

To fully verify the requirements of a block or a system, common practice is to construct a custom testbench targeted solely at meeting these demands. However, unless a standard approach is taken, block-level testbench components or functional coverage elements cannot easily be reused at the system level.

One technique that works well is using a transactor in place of a processor (CPU or DSP) when doing system-level verification. A transactor presents a direct cycle-level accurate mechanism for driving stimulus into the system without the overhead of programming a processor to perform this task. This transactor provides an easy-to-control bus master. This approach also avoids the verification of both master and slave bus agents in one environment, which can lead to challenging error detection conditions.

For example, if there were an error in the connection of a master or slave bus agent, then either could be masking the connection issue. A transactor will also be easier to synchronize with other events in the system design or verification environment. Finally, a transactor can be written to generate a much wider range of protocol values and behaviors over a smaller number of bus cycles, and can therefore be used to achieve better functional coverage.

On the other hand, a transactor cannot directly execute the embedded code written for a CPU or DSP. Thus, detailed processor models are commonly used in software-driven, system-level verification environments. The VMM for SystemVerilog provides guidelines for setting up such an environment to perform software integration and verify the interactions between hardware and software. In fact, a typical SoC project has several different system-level verification environments. For a given system design, its associated functional verification plan specifies system-level verification requirements.

It is easier to use a number of different verification environments to meet these requirements. Each environment is concerned with a particular set of orthogonal functional verification goals and is designed to cover those requirements efficiently. The orthogonality of the system-level requirements makes using a single environment needlessly complex and can in fact mask certain system configuration errors.

In addition to the software test environments, the VMM for SystemVerilog defines four types of verification environments:

The "system validation environment" ensures that the overall performance requirements—such as latency and bandwidth—of the system are met. In this environment, verification components drive or monitor the external interfaces of all blocks in the system. As shown in Figure 3, only the CPU/DSP is replaced with a transactor. This environment should use an abstraction model of the system that efficiently simulates with high throughput and the required level of accuracy to help minimize test turnaround time.

Figure 3 — The system validation environment must be efficient to measure system performance.

System-level verification environments should reuse block-level verification components where applicable. The guidelines outlined in the VMM for SystemVerilog make it possible to architect block-and system-level verification environments so that the effort spent in creating the different block-level verification components is not discarded when moving to the system level.

Verification components constructed as per the guidelines can readily be reused at the system level when required. However, instead of being directly controlled, block-level components can be reconfigured to let them perform operations that are more relevant to the system. The re-configurability of verification components addresses the difference in purpose between block- and system-level verification.

Transaction-level models

The system-level verification methods described in this article often entail modeling some portion of the DUT at the transaction level. Writing a transaction-level model of the entire design is an intrinsic part of some development methodologies. For others, the first model of the design is written at the RTL level, and writing a transaction-level model of the design appears to be a costly activity that will detract from the actual design. But writing a transaction-level model, if the RTL code is not yet available, is an investment — not a cost — that repays itself many times over the duration of the entire project.

A properly written transaction-level model only takes a fraction of the time it takes to write an RTL model, thus enabling the design and the verification team to work in parallel. It also executes orders of magnitude faster than the RTL model, allowing the verification environment and tests to be developed and debugged much faster.

When the RTL model finally becomes available, all components of the verification environment are already at a very mature level. They can immediately verify the functional aspects of the design and can meet high functional coverage for these aspects.

Transaction-level models are faster to write and simulate because they do not have to deal with the intricacies of physical signals and protocols. They can receive, execute and respond to transactions as high-level transaction descriptors.

They do not need to exchange or operate on transactions as a collection or sequence of bits on some physical wire. However, having the ability to substitute an RTL design block in a system with a pin-accurate transaction-level model will enable simulation to run much faster and with fewer demands on the simulation resources.

Design methodologies that require or produce a transaction-level model of the design often use SystemC as the modeling language. It is an excellent choice, and there are no reasons to use a different modeling language because the SystemC model may be used in other contexts than functional verification of the implementation.

But for those design methodologies where a transaction-level model is written only to accelerate the development of verification environments, with no need to reuse that model in other context, SystemVerilog can also be an excellent modeling language choice. SystemVerilog provides all of the necessary high-level constructs that make writing transaction-level models more efficient.

Conclusions

When coupled with an appropriate methodology, SystemVerilog provides all the constructs and features necessary to build a complete system-level (ESL) verification environment. The VMM for SystemVerilog describes several different forms of environments that can be used to verify transaction-level models of the design, the design RTL, and any embedded software. Nearly 100 recommendations and guidelines for system-level and software verification translate the authors' expertise into specific steps for success.

The final article in this four-part series will discuss strategies for adoption of the recommended methodology, including use of the standard building-block and assertion libraries defined in the VMM for SystemVerilog. These libraries provide the basis for the approaches discussed in this article, including XVCs, the XVC manager, and software verification. Taken as a whole, this series of articles will provide a broad overview of using SystemVerilog for the verification of complex SoCs.

Editor's note: Future articles in this series will appear over the next few months.

The first article in the series provided an introduction. The second article in the series focused on RTL verification.

Thomas Anderson is Director of Technical Marketing at Synopsys, Inc. His responsibilities include assertions, coverage, formal analysis and RTL checking. He holds an MS in Electrical Engineering and Computer Science from MIT and a BS in Computer Systems Engineering from the University of Massachusetts at Amherst.

Janick Bergeron is a Scientist at Synopsys, Inc. He is the author of the recently published book Writing Testbenches using SystemVerilog, which builds on the success of its best-selling predecessors, and the moderator of the Verification Guild. He holds an MBA degree from the University of Oregon, a Masters degree in Electrical Engineering from the University of Waterloo, and a Bachelor of Engineering from the Universit du Qubec Chicoutimi.

Eduard Cerny, Ph.D. (McGill University) is a Principal Engineer, R&D, in the Verification Group at Synopsys, Inc. He joined Synopsys in 2001 after 25 years in academia, as Professor of Computer Science at the Universit de Montral. His interests have been in design, verification and test of hardware, and he is author of many articles in these areas.

Alan Hunter, BEng(Hons), MSc, is the Design Verification Methodology Programme manager at ARM Ltd and leads the design verification methodology work for ARM worldwide. This work covers all areas from CPU design verification through systems and system component design verification. His main areas of interest include optimizing design verification efficiency and quality, formal methods, and determinism in the design verification flow.

Andrew Nightingale, BEng(Hons), MBCS CITP, is a consultant engineer at ARM Ltd. and has led the SoC Verification group in ARM's Cambridge and Sheffield design centers for several years. The group covers ARM PrimeXsys platforms and Prime-Cell development, including advanced AXI- and AHB-based system backplane components such as bus interconnects and high-performance memory controllers.