Design Article

Debugging software/firmware using trace function re-usable components

Anindya Dutta and Tridib Roychowdhury

6/1/2009 8:42 PM EDT

One of the challenges in real"time systems, especially in multitasking OS based implementations, is defect fixing. To resolve the defect one has to be aware of the program flow during the defect or faulty condition. Normally, this is done by using in-circuit emulators (ICE) along with the break point feature available in the environment of the emulator.

However, the ICE support may not always be available for the system under consideration. Considering the case that ICE is available for the system under consideration, and the ICE hits a breakpoint, then the dynamics of the system is lost.

All the interactions the system was having with some networks or other systems/modules are stopped at once. Therefore it again may be difficult to reproduce the defect or anomaly exactly. Though some makes of ICE provides a feature of real-time trace function, but that is also limited by the ICE memory.

At times it is required to stop the system in some logical trigger, so that the condition of the system can be analyzed at that point. This feature is also not common in normal debugging systems.

Finally some development systems from WindRiver, tools like WindView offer powerful debugging mechanism. This mechanism allows user to track code flow and task/interrupt scheduler switching in a GUI based interface. However, the development system should have support for deploying this debugging feature.

The method that we propose in this article overcomes numerous such problems with no extra investments. This requires some amount of free memory on board for both code/data and substantial amount of processing capability.

The function trace re-useable component
The method we have developed can be applied to debug the software/firmware of any type of real-time system and is named as "Function Trace" (FT).

In this method, all the functions in the program are uniquely coded and as the function is executed the unique code is noted. These values are stored in a circular array in round-robin fashion. This is called as function code logging. The array contains the program flow information which can be used for debugging purposes.

The basic requirement of employing this method is to have moderate speed processor with some free memory. Thus 32-bit processors are the best choice for applying this method of debugging. Depending on the processor speed this debugging method may be looked upon as an over head for low speed / low end systems.

The tracking capability of the functions executed in the system makes it best suitable for both OS based multi-thread system and as well as OS less systems. Function Trace can also be deployed in multiple locations within a given function to figure out about the program flow within a function just as we might use it for program flow within a module / thread / task.

Function Trace can also be added on with time stamp provided the system considered has a counter or a Real Time Clock on board. This enables us to figure out not only the program flow but also the time taken in various functions / modules within the system.

One of the important feature of this debugging method is that it allows the user to trace only the function(s) which they intended to trace. Therefore this mechanism eliminates tracing functions which are not required, resulting into optimal usage of memory for storage of tracing data. Therefore this helps the developer to focus on the required sections of the code.

Function Trace's basic operating principles
The Function Trace feature (Figure 1 below) creates an execution log of every function call defined in the application firmware. As any application-defined function is called, an entry for the current task/interrupt context (for multi-tasking system) and function called is generated in the log with the help of a unique 32-bit identifier code.

Figure 1: The Concept of Function Trace

The identifier can be generated by concatenating a function code and file code, where the function is defined.

To control the logging of function trace better, the functions can be divided into various levels thus allowing the control of execution flow log. The entire application might be, for example, divided into numerous small applications. The smaller applications might be divided into modules and further into units.

Thus one might think of tracing at the module level functions, some might think of module level or unit level functions for zooming into the problem area. To implement this control, a function level flag is used. All the functions are categorized into some levels. Based on the flag (set/ reset) the function will be logged to the trace for a given level.

The logged data shall be stored in RAM within the embedded system, therefore the user of this method has to plan a way to retrieve the function trace data through communication ports or ICE connectivity or any other method suitable for the system under debug. After the data is extracted, further analysis can be done upon the sequence of execution of the code during the faulty condition.


Next:




spartanwins

6/4/2009 12:50 AM EDT

If gcc is being used, then '-finstrument-functions' can be used to achieve what this article is trying to achieve. In that case the user can just fill up the two standard functions: __cyg_profile_enter and __cyg_profile_exit. Function address is already passed in by default to these functions, so there is no need to maintain a function_name -> function_code mapping.

Sign in to Reply



figurassa

6/8/2009 9:20 PM EDT

Thanks for the tip spartanwins . I was actually looking for this info.

Sign in to Reply



DSP Engineer

6/11/2009 11:59 AM EDT

With the same method and when timestamp is available you can use, in place of MsExcel, rtev.sourceforge.net to display your log as chronogram.
This is very useful: the analyse of text log is boring unlike a graphical view.
Furthermore a software/firmware is close of hardware, a chronogram is more suitable to analyse sequencing, timing or real time problems
So you can locate quickly the anomaly.
The debugging is perform quickly.

Sign in to Reply



Kubsztal

6/23/2009 12:21 PM EDT

Convert trace output to vcd format and use free GTKWave for visualization. Best tool I've ever used for such analysis.

Sign in to Reply



Please sign in to post comment

Navigate to related information

Datasheets.com Parts Search

185 million searchable parts
(please enter a part number or hit search to begin)

Feedback Form