test

A Virtual Test Revolution By making test debugging and verification more concurrent with the design process, virtual testing could dramatically reduce postsilicon test debugging time. by James F. Boettcher and H. Glenn Carson, Jr.

As part of Motorola's commitment to a 10x reduction in development time every five years, we've been searching for new ways to cut the development time of our microcontroller designs. One area that needs improvement is test development--specifically a reduction in the significant time required to debug and verify the test patterns once we receive prototype ICs. With that requirement in mind, we were determined to make test debugging and verification more concurrent with the design process.

The traditional test debugging process requires 25 to 75 percent of the back-end product development effort and by necessity occurs after first silicon. Test emulation provides an alternative, but most existing test emulators aren't particularly useful for complex debugging tasks. Though they emulate the test program, they include no mechanism for interfacing with the device model to provide realistic device responses.

To rectify not only those delays but also the frequently strained communication between the design and test teams, we decided to pursue virtual testing. Aware that virtual test exists for mixed-signal design, we reasoned that a similar approach could be applied to digital design to eliminate the lengthy postsilicon test debugging process we're forced to endure. Our subsequent evaluation has shown that virtual testing can cut the costly postsilicon test debugging time by as much as 50 percent, paying for itself with the savings seen on our first design. It also confirms that moving the test debugging process before first silicon can dramatically enhance our product process through improved design and test cooperation.

The birth of virtual test

We were so enthusiastic about the idea of virtual test that we decided to help make it a reality by working with Integrated Measurement Systems, Inc. of Beaverton, Ore. Together, we agreed to create a tool that could combine the simulation information on the IC with a model of the target tester environment (see Figure 1). After discussing the idea with IMS in 1996, we helped fund the development of the first viable digital virtual test capability. We obtained management backing by predicting that we could cut postsilicon test debugging time in half, allowing us to use valuable testing resources more effectively.

Figure 1	The Digital Virtualtester
The Digital Virtualtester provides a full behavioral model of the target ATE, including full integration of the device to be tested.

Our main goal was a simulation environment that would allow us to load and simulate vector sets, then see the results. The virtual tester could emulate the tester running the vector sets on the particular microcontroller IC and report back results like an actual tester--a process that would work well with our incremental approach to building test vectors. As soon as the design created a stable simulation model, the virtual tester could debug the test set. Just as importantly, we could also verify the tester patterns, thereby helping us to better meet our aggressive testing requirements.

We anticipate multiple benefits from adopting virtual test. First and foremost, it would allow us to push test debugging and verification ahead in the design cycle, making it more concurrent with IC design (see Figure 2). We can also verify some of the DFT features, such as scan, that are typically the least verified part of a design prior to silicon. Virtual test allows test engineers to verify assumptions about the design at a point in the design cycle when it's easy to fix errors. Moreover, moving the debugging of test software earlier in the design cycle frees them to spend more time at the front end of the next project, verifying the test patterns and finding potential test issues earlier in the cycle. That benefit leads to improved pattern quality and shorter cycle times.

Overall, virtual test significantly reduces postsilicon debugging time and lowers costs by minimizing the need for using expensive testers and prototype ICs for vector debugging. Because test debugging and verification would occur concurrently with design, test engineers can spend more time troubleshooting. Instead of scrambling to debug the test under intense pressure, test engineers can fully debug and verify the patterns while the IC is being finalized and fabricated. Test engineers can also capitalize more directly on designers' know-how: Both groups work on the same design simultaneously, improving communication between them and eventually leading to higher first-pass success rates.

Testing the tester

Once we received the first beta model of the virtual tester product, Digital Virtualtester, in late July 1997, we were eager to put it to work. We knew, though, that we had to make a full assessment prior to implementation in order to protect our extensive investment in our existing design flow. So we committed the resources--from both design and test--to thoroughly evaluate the tester to ensure that it indeed offered a viable approach.

Figure 2	Cycle compression with virtual test
Virtual test allows test debugging to begin before first silicon, with a corresponding acceleration of release to production and availability.

The first round of acceptance tests took 10 weeks. We initially conducted a blind test by downloading 20 sets of test patterns--some clean and others faulty--and asking the team to separate the good from the bad. The engineers immediately found an error on the first set of patterns, which was supposed to be a good set. After investigating, though, we discovered that the pattern actually was faulty--for we had inadvertently used an earlier, buggy version of the pattern. After that false start, the engineers quickly moved through the remaining patterns, obtaining the correct results.

We then expanded the number of patterns and achieved excellent correlation between the actual testing results and the virtual tester output. Then we performed the same in-depth evaluation on another IC design that was postsilicon, again obtaining good correlation between the simulated and actual test results.

We also used the prototype virtual tester to help accelerate debugging on a test program that was currently in development. It had taken us weeks to track down a problem in the test set. Impressively, on the very first run the virtual tester revealed the same error in the test vectors, dramatically demonstrating how a virtual tester can literally save weeks of frustrating effort.

During the exercise, we found that the virtual tester gave a much better view into test vector activity than did the actual testing environment. We could examine the activity of any node of interest during the test, just as if we had a microprobe on the device. We thus had invaluable insight into the inner workings of the IC's functionality, which dramatically expedited the debugging process. With the virtual tester, we could quickly trace a problem back to its source--either an incorrect translation of the original Verilog simulation or an actual design problem hidden by other problems in the verification test bench.

The virtual tester also proves helpful in postsilicon processes. In another case, we used the virtual tester to debug a test pattern for an airbag peripheral device that was already postsilicon and under evaluation by a product engineer. Almost all of the vectors passed, except for a few where the clock prescalar was set to certain values, making the part look as if it were out of sync with the vectors by a few cycles. The product and verification engineer first assumed that the problem arose from some asynchronous logic and started looking at ways to "fix" the vectors to match the part.

When the engineer brought the test team in to help, we decided to evaluate the device on the virtual tester to see if the problem would also occur there. A relative newcomer to our team pinpointed the source of the problem in a couple of days. It turned out that the pattern preamble--the initial vectors designed to put the part into a known state--clocked the design less often than the Verilog simulation did after reset. That case is a perfect example of the Digital Virtualtester's usefulness in debugging in the simulation environment to find problems that are difficult to debug on the tester.

The virtual tester also proved extremely useful when an ATPG tool's expected output didn't match the silicon. Our DFT engineer spent weeks struggling to simulate the pattern using the tools from the ATPG vendor. The Digital Virtualtester allowed us to simulate the pattern after just a half day's work, saving us many hours of tester time. The simulation verified that the scan violations in the model would have to be fixed in the next silicon revision before we could fully use scan on the design. That particular experience demonstrated how the virtual tester can also serve as a tool to "verify" other tools used in test program generation.

We strongly recommend that test engineers be familiar with Verilog before using the virtual tester--as they will have to deal with issues concerning the separation of the model from the test bench--and with Verilog errors and debugging of the initial setup. Advanced Verilog knowledge, as well as training and experience in the simulation environment, are needed to integrate the virtual tester into the tool set for a particular design. Fortunately, the flows for both the virtual tester simulation and the test bench simulation are very similar, facilitating the integration of the Digital Virtualtester into a simulation environment.

Final analysis

We couldn't have been more pleased with the results of the Digital Virtualtester's evaluation. All indications confirm that it definitely shortens the time to test. It also provides full presilicon debugging capabilities and supports in-depth verification to ensure a robust test suite. It proves to be useful even for postsilicon debugging.

Our preliminary results give us confidence that we can drive postsilicon test debugging time below our original 50 percent estimate. Because the virtual tester allows our test engineers to debug a problem at any time, we can devote more product and test engineering time, after silicon, to evaluating, debugging, and characterizing the silicon--not the test program.

More importantly, implementing virtual test also lowers testing costs. We can deploy postsilicon engineering resources--both test and design--more effectively. Equally important, instead of using multimillion-dollar testers as debugging tools, tying them up for weeks, we can use the testers in the role they were designed for--the testing and characterization of silicon. And since virtual test provides more robust, higher-quality tests, first-pass success rates will rise--an important goal for any design group in today's competitive markets.

Jim Boettcher joined the Semiconductor Products Sector of Motorola, Inc., in Austin, Texas, as a test engineer in 1979. He's currently the test engineering manager for the Powertrain Systems Division and is a member of the technical staff.

H. Glenn Carson, Jr., is a member of the technical staff in the Transportation Safety and Chassis Systems Global Design Operation divisions. He joined Motorola's Semiconductor Products Sector in 1984 and has been working in test on a variety of microcontroller designs.

To voice an opinion on this or any Integrated System Design article, please email your message to miker@isdmag.com.

integrated system design  December 1998