Introduction to and Regression Test for OCP SystemC Channel Models

We then develop test cases for this environment. The test cases consist of specific requests to be issued by the master and the responses to be returned from the slave. Figure 6 shows the concept of the regression test for the channel models. We assume that a previous version of SystemC OCP Models has been well tested and verified, and hence we can execute our test cases on these models to obtain some important information that can serve as golden results. The information to be verified by this regression test includes the issue time, acknowledge time, the date and the address of each request from a master, and the issue time and accepted time from a slave.

When a test case is executed during the regression test the related information is recorded in a file. This information can then be compared with the golden results to determine whether any functionalities have been violated due to the modification of channel models.

6. The concept of the regression test methodology.

Test Cases and Test Mechanism
We divide the test cases into two groups: direct test cases and random test cases. The direct test cases are developed by the SLD WG of the OCP-IP to "directly" verify those codes that have been modified. The authors of the modifications can easily predict the results of direct test cases since the purposes and results of these test cases are well defined. However, for a complex design it is impossible to directly test all possible inputs and thus random test is frequently used to cover those corner cases. In this article we also created random test cases in order to provide more complete test results.

Directed Test Cases
We leveraged existing directed test cases from previous versions. There are 51 directed test cases and each of them includes a master, a slave, and an OCP channel between them that is exactly the same as that shown in Figure 6. For each test case the OCP channel is configured into an appropriate configuration based on the information provided in a configuration file associated with the test case. Thus different test cases can be used to verify different channel configurations that have been specified.

Random Test Cases
We developed the random test mechanism based on the examples attached to the SystemC OCP Models released by the OCP SLD working group. The examples include the environment for two models to communicate with each other through OCP channel models. They show how the master and the slave communicate with each other. We utilize the examples to construct the environment for the random test mechanism which allows the channel model designer to construct their own random test cases in a controlled fashion defined by themselves such as normal distribution or Poisson distribution.
The random test cases also consist of requests and responses. However, we regard the OCP channel models as a black box and produce some random communication requests between the master and the salve. The random test set currently includes 24 requests. What we concern here is the status of the channel during the communication. We hope that the random test suite can activate some corner cases that are not exercised by the directed test cases.

The status of OCP channels is recorded during a regression test, which includes some information described in Section 2. It should be mentioned that we also adopt OCP channel monitors to help us trace and record the status of the channel.
Next we show how the master model issues the requests in random test cases. As long as we know how to issue requests with the master, we can activate the communications between the master and the slave. Figure 7 and Figure 8 show a typical example of the codes in the master model as explained below.

For the purpose of easy explanation, both Figure 7 and Figure 8 are represented in a matrix form with four columns and six rows. Each pair of corresponding elements of the two matrices represents the issue time and the type of a request. For example, the elements of the first columns and the first rows in Figure 7 and Figure 8 are "20" and "OCP_MCMD_WR," respectively, which means that the master will issue a write request at the time of 20ns. Similarly, Figure 7 and Figure 8 indicate that the master will issue a read request at the time of 25ns, 5ns after the first request. We utilize the two matrices to arrange what request will be issued at what time. Figure 9 illustrates the timing information of the first three requests.

9. The time information of the first three transactions.

The channel model designers can utilize the above two matrices to construct their random test cases. For example, we can fix all the elements in the command matrix and make every element in the time matrix a random one. It means that we can produce the elements in the time matrix with a random function. By giving a time range and a specific distribution function, we can generate all the elements in the time matrix in a desired random manner.

Test Results
Before the SystemC OCP Models v2.1.3 was released we ran all of the direct test cases and random test cases under the regression test environment. We first ran the direct test cases on channel models v2.1.2. The golden results are already produced and attached in the SystemC OCP Model v2.1.2 package. It was expected that the results we got from running the test cases on v2.1.2 were consistent with the golden result. However, we found that the results were not the same as the golden ones. It appeared that some inconsistency existed in the system before our regression test was applied. We later ran the directed test cases on the channel models of v2.1.3 which were to be released. The results were identical with what we ran on v2.1.2. Therefore we doubted that the original golden results might contain some problems. After consulting the SLD WG who developed the v2.1.2, we were told to replace the golden results in v2.1.2. Then, we ran the random test cases on the previous version (v2.1.2) to get the golden results. The simulation cycles are 1,437 ns. Then, the random test cases run on the OCP channel models of v2.1.3, and the simulation cycles are the same with the previous one. We checked two files produced from the simulation to verify the modifications on the new version. One file contains the 7 items of information which we have explained in section 1, and the other records the status of the channel. Both of the two results are consistent with the golden results.

Acknowledgement
This work was supported in part by the National Science Council under contract No. NSC95-2220-E-006-011.

References

1. Janick Bergeron, Writing Testbenches: Functional Verification of HDL Models, 2^nd ed, Kluwer Academic Publisher, 2003.
2. A. Bernstein, M. Burton, F. Ghenassia, "How to Bridge the Abstraction Gap in System Level Modeling and Design," in Proc. of International Conference on Computer Aided Design, pp. 910-914, 2004.
3. Ali Habibi and Sofiene Tahar, "Design and Verification of SystemC Transaction-Level Models," IEEE Transactions on Very Large Scale Integration Systems, pp. 57-68, 2006.
4. Susan Xu and Hugh Pollitt-Smith, "A TLM Platform for System-on-chip Simulation and Verification," in Proc. International Symposium on VLSI Design, Automation and Test, pp. 220-221, 2005.
5. M. Caldari, M. Conti, M. Coppola, S. Curaba, L. Pieralisi, C. Turchetti, "Transaction-Level Models for AMBA Bus Architecture Using SystemC 2.0," in Proc. Design, Automation and Test in Europe Conference, pp. 26-31, 2003.
6. Sudeep Pasricha, Nikil Dutt, Mohamed Ben-Romdhane, "Extending the Transaction Level Modeling Approach for Fast Communication Architecture Exploration," in Proc. Design Automation Conference, pp. 113-118, 2004.
7. OCP-IP Website, http://www.ocpip.org/home
8. Lukai Cai and Daniel Gajski, "Transaction Level Modeling: An Overview," in Proc. International Conference on Hardware/Software Codesign and System Synthesis, pp. 19-24, 2003.
9. Anssi Haverinen, Maxime Leclercq, Norman Weyrich, and Drew Wingard, "SystemC based SoC Communication Modeling for The OCP Protocol," OCP-IP Website, 2002.
10. Open Core Protocol Specification 2.2, OCP-IP Website, 2006.