Before explicitly defining the term "simulation graphics", it is helpful to look at an example that illustrates the need for simulation graphics.

An automobile company’s marketing department determines that a "smart" cruise control system will differentiate it from the competition. This system has radar to detect the position of other cars and is meant to maintain safe following distances while maximizing speed. Unlike its predecessor, this new cruise control can automatically slow down the car and even apply the brakes.

These new capabilities seem straightforward enough, but they affect a notable number of sub-systems in complex, untested ways. Besides requiring a significant rewrite in the cruise control software, this feature affects the engine control system, the braking system, the suspension, and countless other components and electronics. The people involved in the design and implementation of this new feature include industrial designers, embedded programmers, electronics engineers, mechanical engineers, control systems engineers and even technical writers.

Because of its complex nature, the engineering team decides to create a complete system-level model of the new feature and the affected sub-systems. The engineers start the modeling process using the written specifications from the marketing department. They simulate the design, find defects and rapidly fix them. After six months of effort, each of the participating engineering teams feels comfortable with the results.

Before finalizing detail specifications to be used in the implementation process, the engineering team decides to check the model with the marketing department, company executives and a few prospective customers. They start by showing the marketing department their block diagrams, graphical plots and statistical analysis of simulation results. Because of the abstract, dry nature of this representation, the marketing manager becomes confused quickly. He provides no useful feedback. The marketing manager tells engineering to proceed by building a physical prototype.

Unsatisfied with the prospect of building an expensive physical prototype without adequate initial feedback from marketing, engineering builds an interactive, photo-realistic graphical model of a dashboard that looks and behaves like a car’s real dash. Figure 1 shows this view and the attached control system model. They add a few other cars to simulate road conditions and connect this graphical model to their simulation. The entire "virtual prototype" is sent back to marketing. It is immediately put through its paces and within five short minutes, a problem is found. Someone hits the brakes and expects the cruise to turn off completely. Instead, the cruise control begins to accelerate the car after the brake is released. The driver of this virtual vehicle crashes into another car.

The team analyzes the cause of the problem. The marketing department never explicitly accounted for this case in the written specifications, so the software engineer made some assumptions based on his own experience. "Of course you would want the car to accelerate under these conditions. Why else would you want a smart cruise control?" At this point, the programmer asserts that the marketing manager was wrong to expect the cruise control to turn off under these conditions. The marketing manager argues that it does not matter whether he was right or wrong. The fact remains that he crashed the car, and it is likely that other drivers would do the same. The programmer agrees.


Figure 1: Simulation graphics for a cruise control model.

More testing is done and twelve equally critical problems are found. The specifications are rewritten and the model is fixed. A total of one month is spent on the fixes. The team estimates that the fix would have taken nine months if the problems were not found at the early modeling stage using simulation graphics. In addition, it would have cost over $250,000 to build the hardware prototype needed to find the same bugs. More importantly, the amount of revenue lost during this extra eight-month period would have been several million dollars.

This example illustrates an essential benefit of simulation graphics. You can find mistakes early in the design process. This saves money and reduces cycle time.

Modeling and Simulation Tools

Simulation graphics software is used to enhance the value of modeling and simulation tools. Simulation graphics do not stand alone. For this reason, a short discussion of the available modeling and simulation tools is necessary.

Figure 2 is a chart of the simulation and modeling tools for embedded systems development. From left to right the tool chart covers the engineering disciplines involved including mechanical, electrical, system and software design. From top to bottom the chart spans the range of high-level design tools to low-level tools used at the implementation stage.

Figure 2: Embedded system modeling and simulation tool chart.

Manufacturers have used low-level tools as a standard part of the development process for many years. CAD/CAM software, board layout tools and software compilers are now mainstream products. These tools are necessary to build embedded products. The high-level tools are not yet necessary, but those who use them gain a significant competitive advantage. As systems become more complex and competition strengthens, these tools will change from being optional to necessary.

System simulation tools help manufacturers understand and design very complex systems that integrate hardware and software. Their most important contribution is to create an executable version of the specifications, without the cost of building a physical prototype. This executable specification is often referred to as a "virtual prototype". The virtual prototype is valuable because, unlike a physical prototype, it is relatively inexpensive, quick to change and easy to re-test when an error is found.

While the value of these system simulation tools is high, they have serious drawbacks. They can be very complex, and those unfamiliar with them, such as managers and customers, do not typically understand their operation or results. The purpose of simulation graphics is to make the benefits of these tools accessible to the customers and managers who must make the critical decisions about the product’s development and marketing direction.

Simulation graphics software links to the high-level design tools because it is the first representation a customer sees. However, the value of graphics can be further enhanced if it also links to low-level tools and various tools across engineering disciplines. This provides a common view to the customer and enables high-level testing at the low-level implementation stage of development.

Simulation Graphics

To understand the benefits and purpose of simulation graphics, we must first define the term. Simulation graphics are the graphical user interface components for viewing simulation data and interacting with the simulation model. By simulation graphics, we do not mean the graphical block diagrams used to build the basic model structure. Nor do we mean the GUI that’s used to build this model. Simulation graphics are classified into four major categories.

  1. Simulation Data Display
  2. Simulation User Interface
  3. System Visualization
  4. Virtual Front Panel

1. Simulation Data Display

Simulations produce a large amount of data. The more complex the model, the more data produced. The usefulness of these simulations not only depends on the completeness and accuracy of the model, but also on the ability to view the resultant data in an organized, convenient and comprehensive manner.

Figure 3: Simulation data display components including plots and strip charts.

Data display graphics show this data in several forms. They include plots, strip charts and numeric displays. These are "output-only" graphics, that is, the user simply views them. The user does not add input that affects the simulation. Data display graphics can be viewed in real-time or post-processed when the simulation completes. Figure 3 is an example of such a display.

There are several benefits to a well thought out graphical data display screen. First, when viewing data in real-time, information is displayed quickly. A properly organized data screen allows the user to see bugs and optimizations at a glance, even if the data is scrolling by rapidly. Second, data is not only important by itself, but also in relationship to other data. For example, a falling tachometer by itself is not abnormal. However, if at the same time, the speed is increasing and the accelerator is being pressed, then you know there is a problem with the model. A well-designed data display can help you catch these interactions. Third, the number of interesting data streams can grow rapidly until, eventually, you have multiple pages of data displays. The advantage of a good data display package is that it allows you to collect your most important data and arrange it in one or several well designed screens for easy viewing and navigation. With the additional use of audible and visual alarms, colors, line styles, large and bold fonts, etc., we can catch error conditions or optimization opportunities that otherwise would have gone unnoticed. The ultimate result is a shortened development cycle because mistakes in the model and specifications are found as early as possible.

Data display graphics are typically for internal use only. They are a debugging, optimizing and design tool. Although in some cases, a well designed data display screen can be used to communicate results to those who are not using the modeling tool directly. This type of communication, however, is usually reserved for the photo-realistic, virtual front panel stage of simulation graphics. We will discuss this later.

Most modeling and simulation tools come with a basic set of data display components. These have the advantage of being tightly integrated and easy to use. They have the disadvantage of limited functionality and are often difficult, if not impossible, to extend. In this situation, the modeler must create his own display software from scratch or use a third-party graphics package. If he is displaying real-time graphics, this package must be integrated with the simulation tool. If the data is being post-processed, the graphics package must be able to read from files.

An advantage of going to a third party graphics package is that the graphics displays can be re-used by other members of the development team who are using different simulation tools. The basic built-in graphics that are packaged with the simulation software, on the other hand, can only be used with one tool. In addition, having a common screen shared by different parts of the design process allow for a consistent view of the data. This consistency is helpful as subsystems are integrated to create the complete embedded product.

2. Simulation User Interface

All modeling and simulation software comes with some type of Graphical User Interface (GUI). These GUIs allow the modeler to create the graphical diagram and set parameters of the models. These GUIs are typically general purpose and relatively complicated to learn; therefore, only the modeling expert is qualified to change the model or its parameters. Quite often, however, it is desirable to have an application expert, who is not the modeler, make changes to the model and its attributes so he can perform "what-if" scenarios to test and optimize the system.

A high-level simulation user interface allows the modeler to create a "model specific" GUI that others can use with little or no training. It allows the application expert to set parameters and make limited changes quickly and easily, without learning the modeling tool. This saves significant time because the application expert does not have to wait for help from the modeler. It also allows the application expert ample time to "play" with the model without worrying that he is tying up the modeler’s time and resources. This "pressure-free" environment results in better, more complete model testing. Figure 4 is an example of a simulation user interface that can be used by a application expert to optimize conditions.

Figure 4: Simulation user interface for an automobile engine control system.

Modelers can also use the simulation user interface to log events created by the application expert. This provides an elegant and efficient method of generating regression tests. This simulation user interface often takes the form of simple, off-the-shelf input or control components such as sliders, dials and numeric input objects. They have the ability to work in run-time as well as the development environment. The simulation user interface can even be run over the web while the simulation is being run on a server in a central location.

3. System Visualization

Most embedded products work as part of a larger system, so the design of these products requires a simulation of the entire system, not just the stand-alone product.

Figure 5: System visualization of an airplane flight control system.

In these cases, it is useful to create an animated graphical representation of the entire system. For example, a traffic pattern of cars must accompany the car that is being simulated and tested. Traffic lights, streets and pedestrians might also be represented. In addition to interacting with other products, the product itself is nothing more than a collection of sub-systems, which interact and must be simulated and visualized. For example, besides showing the cockpit of an airplane, which provides the obvious view into the airplane’s engine parameters and status, you might want to see an external view of the airplane to observe its pitch or yaw. You may want to see the ailerons and rudder move while you view the artificial horizon in the cockpit. You might also want to see a schematic view of the hydraulic system and light up sections of the diagram as pressure is rising or falling. Figure 5 shows this airplane system view just described.

Unlike basic data displays, system visualizations provide a more custom, application-specific view. An airplane’s pitch is represented by an airplane, which moves on the screen at the appropriate angle. It is not simply a number or plot that represents that pitch. This provides several advantages. First, an application-specific system visualization is much easier for the modeler to understand at a glance. There is less time spent mentally converting abstract representations and more time reconciling different conditions and parameters to determine bugs and optimizations. This could be the difference between catching a bug or missing it. The second advantage is that the application-expert can understand and use the model.

The disadvantage is that it takes additional work to create application-specific graphics. With data display components, you can simply pull them out of a library. Some system visualization components, however, require custom graphics work. This will take some additional effort and the value added must be weighed against the benefits.

4. Virtual Front Panel.

One of the most powerful and dramatic benefits of simulation graphics is achieved by allowing your customers to participate in the modeling and simulation effort. This helps in two ways. First, the customer will find mistakes in the specifications and model that the systems engineers will most certainly miss. Second, it will help solidify the customer’s idea of what he really wants, not what he originally told you he wanted. This is critical, since the goal is to build products that customers will ultimately buy.

Figure 6: Virtual front panel of an airplane cockpit.

Creating simulation graphics for customers is much different than for internal use. The graphics must be much more realistic, accurate and complete. This is for several reasons. First, a customer cannot easily understand abstract representations. The modeler understands the model thoroughly. The customer, however, is less sophisticated and knowledgeable, so nothing must be left to the imagination. Second, the customer’s impression of your product is at stake. Even if you tell them "it is only an early prototype", they will judge your product based on that first impression.

So how do you create impressive graphics without breaking the bank or schedule? After all, don’t these 3-D, virtual reality style animations take years of work, hundreds of programmers and massive graphics workstations to create? No – and the answer lies in re-use and selective graphics design. The important thing is that the prototype looks good, is interactive, has real-time response and is accurate. It does not have to be a Hollywood production.

The looks are actually the easiest part. With today’s plethora of excellent graphics packages such as PhotoShop, Freehand and CorelDraw you can cheaply and easily create high-quality, photo-realistic images. You can also import bitmap images from sources such as scanners, digital cameras and 3-D rendering software. Such renderings are often readily available as part of the early mechanical design and prototyping process. These photo-realistic images provide more than enough graphics fidelity. These images have the look of a 3-D prototype but the interactive speed and ease of manipulation of 2-D graphics. At this point you can add the interaction, behavior and animation to bring the graphics to life. You connect the animated graphics to your simulation model and you have a customer ready virtual prototype. The key is that these graphics can be created quickly and do not require expensive hardware to run in real-time.

When creating virtual front panels for customer use, it is recommended that a graphics or industrial designer help create the graphics images and animations. These people are much more effective at creating realistic looking models, and they are a lot cheaper to use than systems or software engineers. For example, the cockpit display in figure 6 was created by a graphics designer at ¼ the cost and ½ the time of a software engineer. More importantly, the result is much more impressive.