The forgotten SoC verification team

An approach to solving the SoC verification problem
A new method for SoC integration verification considers verification goals and the design of tools that can find valid, but randomized, ways of achieving those goals. Solving this problem requires a focus on the goals and outcomes rather than the input stimulus. In order to solve this problem, the focus needs to shift from stimulus and to goals and outcomes.

Tools should be constructed that can find ways to meet the goals and provide a sample of the possible ways those goals can be achieved utilizing different parts or paths through the system.
The information necessary for a tool to do this can be divided into four main types:

• Register and Memory Map. In order to know how to talk to each IP block, it is necessary to know how they can be accessed by each processor in the system. Software needs to know how the bit fields or registers are laid out and any relationships or dependencies that may exist between them, often described in an IP-XACT description of the system. This information is necessary for driver-level verification.
• Driver Models. While it is necessary to have models for blocks in the system, these do not have to be complete functional models. Instead, sparse models may be used which only describe aspects of the block necessary for SoC integration and verification. Models should be reusable in both a vertical and horizontal manner. Vertical means that a model can be incorporated into a sub-system model and that, in turn, can be incorporated into a larger system model. Horizontal means that generic operations can be defined and quickly adapted for specific needs. This information is necessary for driver-level verification.
• Application Models. What is the system meant to do? A model such as this is ideally developed incrementally. Waiting until all the use cases have been defined to start performing the verification is not necessary. As new use cases are added, or exiting ones refined, new spaces in the device can be exercised. It should also be possible to measure verification coverage by looking at possible paths that have been exercised through this model.
• System Models. As previously discussed, today’s SoCs include a significant amount of functionality at the system level and this is increasing in size and sophistication. Power and clock management can affect all aspects of device performance.

This level of information is sufficient to automate SoC integration verification. It is possible for a tool to automatically generate self-verifying C test cases that are compiled and run on the embedded processors. These test cases verify every aspect of the intended top-level SoC functionality while not repeating the UVM-based testbench verification typically performed at the block level. Multi-threaded test cases stress the SoC RTL design, especially resources such as bus fabrics and memories shared by multiple blocks. This exercise of parallel behavior is also effective at measuring system-level performance under stress conditions.

An example of this approach is the TrekSoC tool from Breker Verification Systems, shown in Figure 2.

The user starts by defining outcomes and then configuration settings, conditions or stimulus that can produce or modify those outcomes. These are called scenario models, and they span the driver, application, and system levels described previously. These models can be built up in a hierarchical manner either through composition of lower level models or through refinement of existing models.

As more information becomes available, higher levels of verification scenarios can be generated. The tool takes these scenario models along with an IP-XACT register and memory map and, for each user-defined goal, finds ways to make that goal happen. It will then pick a random solution and generate the necessary self-verifying C code to turn that goal into a scenario running in simulation. In this way the test cases can orchestrate multi-CPU and multi-threaded verification scenarios.

As shown in Figure 2, the tool also generates the SystemVerilog TrekBox module to interface with existing UVM testbench components on the SoC’s inputs and outputs. This is necessary since some scenarios will entail reading data into the chip or sending data back out.

The TrekBox and an accompanying events file ensure that test cases running on the embedded processors are synchronized with the testbench components. This ensures that data is read into the chip at the right time and that outgoing data is checked for correctness at the right time. The use of UVM is not mandatory; TrekBox can connect to simple bus functional models (BFMs) or to testbench components using any of the popular standards, including the Open Verification Methodology (OVM) and the Verification Methodology Manual (VMM).

Impact on SoC development flow
The impact of adopting this new approach on the verification process is significant. For a start, it eliminates the need for hand-written C tests or diagnostics running on embedded processors. Recognizing the inadequacy of chip-level testbenches, most SoC integration teams handcraft a small set of software tests to sanity-check the full-chip operation, a slow and tedious process. It is hard for programmers to develop the type of multi-threaded stress-tests that TrekSoC generates automatically. Hand-written tests are difficult to modify and unlikely to fully exercise the entire verification space.

Since chip-level testbenches with processor models capable of running programs tend to run slow in simulation, some SoC teams remove processor models and drive sequences of transactions onto the processor bus with a bus functional model. This is similar to how block-level verification is performed using constrained-random techniques; the processor bus essentially becomes just another input/output port for the chip. However, tests that coordinate the processor bus activity with the regular inputs and outputs are hard to construct, especially when trying to emulate multi-threaded and multi-processor test cases.

Fortunately, TrekSoC can also operate in a fully transactional mode, automatically generating the processor bus transactions rather than C test cases. As with the other input and outputs, these transactions can leverage an existing BFM or UVM testbench component for the processor bus. The transactional mode can be used to verify any design, including individual blocks or those with no embedded processors at all. Block-level scenario models can be reused and referenced by SoC-level scenario models in the process of SoC integration verification.

This approach does not entirely replace the need to execute the actual production software on the design, whether in simulation, acceleration, emulation, or on an FPGA-based prototype. This provides hardware/software co-verification and may also serve as a platform for software development.

However, production software is usually not ready before SoC tapeout, so the integration team may exercise incomplete functionality. Further, production code tends to be well behaved, not exercising much corner-case behavior. Running self-verifying C test cases will better stress the design and find most, if not all, bugs that are otherwise found only in co-verification or even after chip fabrication.

TrekSoC has benefits beyond the immediate advantages of verifying the current chip. This approach can work at all stages of the design and development flow. The verification task is no longer relegated to an end of RTL development activity, as it has been in the past, since it can be applied to models at any level of abstraction. If the SoC team has created a virtual prototype for software development, then the self-verifying C test cases can be run long before the RTL design is ready.

Bringing the integration verification earlier in the development process allows tasks such as performance verification to have a larger impact. It becomes possible to optimize the system for the desired performance and to ensure that the correct controls exist within each block for proper power management. Finally, performing verification on abstract models enables a more efficient process with execution speeds often orders of magnitude higher than for RTL verification.

Conclusions
In a world where methodologies are entrenched, it can be difficult to implement change. In a world where the job is endless, there can be equal difficulties in finding an approach that will bring relief. While there are no entrenched methodologies for SoC verification since the existing methods are falling short, a proven solution is at hand. Breker Verification Systems (www.brekersystems.com), as an EDA solution provider, brings automation and power to SoC verification efforts. TrekSoC provides an answer that is on target to address existing verification limitations, a solution scalable today to enable even more powerful solutions for tomorrow.

About the author
Adnan Hamid is co-founder and CEO of Breker Verification Systems. Prior to starting Breker in 2003, he worked at AMD as department manager of the System Logic Division. Previously, he served as a member of the consulting staff at AMD and Cadence Design Systems. Hamid graduated from Princeton University with Bachelor of Science degrees in Electrical Engineering and Computer Science and holds an MBA from the McCombs School of Business at The University of Texas.

---

If you found this article to be of interest, visit EDA Designline where you will find the latest and greatest design, technology, product, and news articles with regard to all aspects of Electronic Design Automation (EDA).

Also, you can obtain a highlights update delivered directly to your inbox by signing up for the EDA Designline weekly newsletter – just Click Here to request this newsletter using the Manage Newsletters tab (if you aren't already a member you'll be asked to register, but it's free and painless so don't let that stop you).