Testing and Debugging DSP Systems, Part 6

Real-Time Embedded Software Testing Techniques
The specific characteristics of real-time systems make them a major challenge to test. The time-dependent nature of real-time applications adds a new and difficult element to testing. Not only does the developer have to apply the standard black and white box testing techniques, but also the timing of system data and the parallelism of the system tasks must be considered as well. In many situations, test data for real-time embedded systems may result in errors when the system is in one state but to in others. Comprehensive test cases design methods for real-time systems must be employed to achieve the proper coverage and system quality. DSP systems fall into this real-time system category. The basic testing approaches for DSP real-time systems involve (Figure 12):

• Task testing – This involves testing each of the system tasks independently

• Behavioral testing – Test the behavior and actions of the real-time DSP system as a result of external events.

• Inter-task testing – This testing phase involves time-related external events. The system behavior is examined as a result of real-world frequency of external events into the system.

• Systems testing – In this final phase, software and hardware are integrated and a full set of systems tests are executed to detect errors at the software and hardware interfaces of the system.

Figure 12 A Real-time DSP testing process

As we all know, testing is expensive. Testing progress can also be hard to predict and the fact that embedded real-time systems have different needs as described above, we need to plan the testing process to achieve the most effective and efficient plan. The key goal is to know what you are looking for and how to effectively locate problems. You must plan to succeed but also manage risk accordingly and optimize and customize the testing process as necessary. The key question to answer is "What are we looking for?"

As you know, the consequences of a bug vary depending on the severity; production stop, critical, nuisance, "nice to have" are all categories of software bugs that demand different forms of attention. How your organization categorizes and manages software bugs will vary but the severity is usually always on this sort of graduated scale.

Many bugs are found in the source code and are referred to as implementation bugs. These result from improper implementation of algorithms and other processing, data bugs, real-time bugs (which is a category unique to real-time embedded systems) and other system related bugs.

Bugs are found in the source code but there are also plenty of sources of nonimplementation bugs that cause damage as well. Sources of nonimplementation bugs include errors in the specification, design, the hardware and even the compiler (remember the compiler is a software program with bugs just like other software programs!).