According to Harry Koehnemann and Mark Coats, who are conducing a class on "The Value of Modeling in Complex System Development (ESC-423)" at the Embedded Systems Conference in San Jose, Ca., such results should not be surprising in the context of what they describe as the nave "field of dreams" approach that is usually taken in applying modeling to the systems development lifecycle.

The traditional approach to systems engineering or analysis for complex systems, said Koehnemann, has sometimes been to produce a "system of dreams". In this "built it and they will come" approach towards users and customers, he said, developers and modelers first build what he describes as a "system of dreams," and then asks the user if it is what they want.

"When they say no, we go back, redesign, and ask again," he said. "This process repeats until either 1) the customer cancels the effort or 2) the user becomes weary of the process and reluctantly accepts the system. "

As opposed to more traditional approaches which often neglect design up front and then ask about behavior near the end, they advocate an approach that engages customers and stakeholders early in the system discovery process and uses information on the system's behavior to drive design and development.

Every system has a phase early in development whose goal is to understand the scope of the system and to design a technical plan to meet it, said Koehnenmann. Various domains have roles for this activity, systems engineer for embedded and real-time systems and systems analyst for IT and data-centric systems are two examples. The result of a systems effort includes:

1) An understanding of the system's behavior (behavioral requirements in a behavioral centric system)

2) An understanding of the system's data and data relationships (data requirements in a data centric systems)

3) A technology solution that realizes that behavior given other constraining requirements (design)

4) A development strategy to create the solution (technical plan). Historically systems engineers and analysts do perform modeling, but it is commonly for technical feasibility, not understanding.

"During this effort, the engineers ensure the solution is technically feasible which might include some studies, pilot efforts, or models to assess and mitigate the risks of the proposed technical approach," he said. "One might model to perform a cost vs. performance tradeoff."

For example, what bandwidth can we support in a particular frequency range. Or model a portion of the system to see if a requirement could feasibly be met with a particular design or technology. These types of models were and continue to be vital to establish the system's technical feasibility, understand cost and performance tradeoffs, and to validate a technical approach.

In most cases, however, he said such efforts fail to create a complete behavioral model of the requirements or of the system's design.

"The behavioral requirements and the technical design need to be communicated to the team members that ultimately build the system," said Koehnenmann. "Historically engineers have used use cases and scenarios to show behavior and block diagrams to show the system's design.

Since these artifacts serve as inputs that drive the rest of the development activities, engineers must, Koehnenmann insists, ensure that they have identified all the behavioral requirements and that their design realizes all those requirements.

"Any items missed during this analysis will cause rework further downstream," he said, "which, depending on the size, can be costly and slip schedules. When using only text to represent behavioral requirements, validation can only occur by painstakingly reviewing textual requirements in peer reviews. "

The same is true for designs represented as simple block diagrams, which are also validated by reviews. Any changes to the behavioral requirements and design must be re-reviewed which is expensive, time consuming and has the potential for missing items that lead to errors downstream. Systems engineers and analysts need a solution to manage the ever growing complexity in behavioral requirements.

In their class, said Koehnenmann, a SysML-based, model-driven strategy is used for specifying and managing both the system's complex behavioral requirements and the system's design. The ability to simulate the system before it is built is one of the most compelling benefits of using system models, he said.

"Simulation is one of the best forms of behavioral validation especially if the customer is involved in the process and can provide direct feedback during model development," said Koehnenmann. "Another benefit is that customers feel more confident when they see user level behavior simulation thru a system model.

"Customers realize how complex today's new systems have become and are more often requiring their contractors to demonstrate they thoroughly understand these complex systems. Executable system models offer a more compelling demonstration of system knowledge than Powerpoint presentations."

Applying modeling to the system development lifecycle can help resolve many challenges, he said, and effectively drive the entire systems lifecycle development, from requirements to design to implementation to validation.

"The benefits demonstrated include more effective communication of system behavior, earlier validation of system behavior - prior to hardware and code being available, tracing between system requirements, design elements, and tests, and finally effectively managing the system's evolving requirements and design."

The approach the two developers advocate, said Koehnenmann, separates modeling system understanding and modeling system design and provides a vehicle to collaborate with all system stakeholders, customer and end users, regarding their expectations on system behavior This collaboration provides the key mechanism for validating system requirements.

"Our community will not be able to meet the ever increasing complexity in behavioral requirements without a new approach," he said. "The method we advocate requires system engineers to change the way they analyze, develop, and validate requirements. The significant benefits will be seen in the backend when we build the system the user expects, not the 'system of dreams, we think they might want."