To make its affinity-based RTOS scheduling algorithm for multicore software both testable and flexible, Green Hills is using LDRA's lightweight code coverage tools.
In the newest implementation of Green Hills Software’s (GHS) Integrity-178 tuMP RTOS, the company has come up with an application environment with the flexibility and responsiveness needed for high-end heterogeneous multicore designs in military/aerospace, automotive, and high-performance computing applications.
To achieve this more flexible approach, Green Hills has moved away from the current symmetric (SMP) and asymmetric multiprocessor (AMP) paradigms now used in most multicore designs. Instead the company’s flagship RTOS is based on affinity grouping which matches code more exactly to specific processors than these previous approaches.
In SMP, a pool of multiple identical cores work share the same memory resources but operate independently, each executing different programs and working on different data, but able to share resources. In AMP designs, not all CPUs are treated equally. In such a configuration a system might only allow one CPU to execute operating system code or might only allow one CPU to perform I/O operations. Complicating the issue in today’s heterogeneous core designs, the two modes of operation are sometimes intermixed, some in SMP and others in AMP.
The limits of AMP and SMP
The problem with traditional AMP and SMP approaches is their rigidity. Using them, a typical implementation during initialization is able to associate a core or set of cores with an application or group of applications but the scheduling for such groupings is not shared between processor sets. While any unallocated core could be used for new applications (or to extend an existing application), these traditional paradigms are at a loss when it comes to utilizing spare capacity on cores assigned to different processor sets.
As a result, in AMP/SMP based designs it can be difficult to run virtual machines having multiple virtual processor cores because there is typically no permanent correlation between a virtual processor core and a particular processor core. This can lead to situations where a virtual processor core is limited to using only features that are supported by all of the physical processor cores, which reduces the benefits of having asymmetric processor cores.
Another problem is that it can be difficult for the operating system and application to manage the heterogeneous nature of the system on which they are running. This can lead to situations where system threads or application threads are scheduled on cores that do not support functionality used by the threads. This can cause functional issues in some situations, and performance issues in other cases, due to attempts to execute instructions that are undefined for particular cores.
To make multicore development more flexible on its Integrity RTOS, Green Hills has replaced use of traditional AMP and SMP methods with a proprietary alternative based on affinity grouping.
In its time-variant, unified multiprocessing RTOS (tuMP), Green Hills has implemented an affinity-based scheduling method that allows the developer to define operations and group them according to their software code affinities, which may vary over time. The algorithm the company has developed allows such affinity groups to be scheduled independent of other affinity groups, permitting time lines that closely correspond to application requirements.
At the same time the RTOS allows multiple other affinity groups to be developed that can make use of any of the time windows where cores are not being utilized. Any new application (or extension of an existing application) can make use of the unallocated execution time across the entire multicore processor.
In addition to allowing the use of affinity grouping as an alternative to either AMP or SMP, the tuMP scheduler also makes possible practical time-variant scheduling of both AMP and SMP applications simultaneously.
Making affinity grouping more testable
While this approach provides the flexibility and speed of response needed, many of those same high performance applications in automotive and mil/aero markets have code coverage requirements that have been difficult to achieve in multicore designs.
Both ISO26262 in automotive and DO-178B/C in mil/aero require compliance at several levels of precision. At one end it can be simply showing whether a line of source code has been executed at least once by a set of test cases. At the other end, it is necessary to use more complex modified condition/decision coverage (MC/DC), where a component is placed under exhaustive testing.
It is not just because such standards require a direct one-to-one mapping between specific code and the core it runs on that makes code coverage difficult on multicore designs. According to Steve Morton, a consultant and FAA Level A Software Designated Engineering Representative, it is much more than that. "Multicore software represents a blurring of the lines separating different pieces of software – they still execute at least semi-independently, but they are more interrelated than more traditional systems."
On the plus side, affinity grouping deals with this issue by providing, at any moment, a mechanism for direct mapping of execution paths to specific cores, thus making code coverage more straightforward. The downside is that many code coverage and analysis tools require the heavy-weight, line-by-line instrumentation technologies that can barely keep up with the dynamic nature of affinity mapping.
The problems with doing safety- and security-critical code coverage in a multicore environment relate to core affinity and the bewildering array of synchronization mechanisms that must be tracked. To provide developers using its time-variant RTOS with a lightweight and flexible way to meet certification requirements, GHS has just partnered with LDRA Inc. to integrate the latter’s tracing technology into the company’s MULTI development tool chain.
LDRA’s trace technology tools make use of branch point instrumentation and aggregation to capture run-time dynamics without significantly affecting the overall performance on any of the cores or on the overall system. Rather than lock down the execution path being analyzed to ensure accurate measurement, in the LDRA approach run-time dynamics are simply captured, aggregated, and piped back to the host platform.
Performing structural analysis of the code on a multicore processor, the LDRA coverage tool flow graph shows exactly what has and has not been “covered” in the execution of the code.
Structural coverage analysis can be captured at the individual core or aggregated to provide a multicore system-level view. Results are portrayed graphically, enabling developers to follow the flow of code execution and to assess system behavior at core and system levels.
Multicore is hard, compliance is harder
Even with such building blocks and tools, doing multicore software apps that require certification compliance will always be difficult. One issue code coverage specialist Morton foresees is how to naturally create the conditions to cause a semi-visible program to follow a specific desired path through the code, which is often blocked because external stimuli may be filtered by adjacent cores’ programs, causing the semi-visible program to operate in an unexpected manner.
He said that from a tooling perspective, the issue comes down to whether the tools can appropriately handle the number of cores and the semi-integrated hardware operating environment. "From a regulatory perspective, the applicant must demonstrate that they have considered the issues, and present a plan for satisfying the objectives."
Join over 2,000 technical professionals and embedded systems hardware, software, and firmware developers at ESC Silicon Valley July 20-22, 2015 and learn about the latest techniques and tips for reducing time, cost, and complexity in the embedded development process.
Passes for the ESC Silicon Valley 2015 Technical Conference are available at the conference’s official site with discounted advance pricing until July 17, 2015. The Embedded Systems Conference and EE Times are owned by UBM Canon.