Mocking the Embedded World - Page 3

Michael Karlesky, Greg Williams, William Bereza, Matt Fletcher
Atomic Object, http://atomicobject.com

Implementing a System Feature – An Overview

Step 1 – Create One or More System Tests

The process begins with creating a system test to verify an aspect of a requirement with a specific test case. Some amount of Systir drivercode will be necessary to establish the language of the system test and providesufficient functionality (e.g. digital I/O, analog voltages, network traffic,etc.) to exercise the hardware of the embedded system. For direct hardwareinteraction, we make calls to our miniLAB wrapper. For data over communication channels (e.g. RS232 or Ethernet) calls to other libraries are necessary.

A typical system test generates a stimulus input and then gathers output from the board under development. Assertions comparing theexpected and collected output will determine whether the system test passes orfails. Of course, at this point in the process, there no production code forthis feature exists so this system test will fail until the feature is complete.The constraints of the system test, however, even at this early stage, willguide implementation and force disambiguation of the system requirement currently driving development.

Step 2 – Generate an MCH Triad and Unit Tests

No production code or unit tests yet exist. Development continues with using the file generation script. A Model, Conductor, andHardware are each generated with a source, header, and unit test skeleton file.

Step 3 – Add calls to the Executor for the Conductor’s Init and Run Functions

A system of even modest complexity will contain several MCH triads. For multi-triad systems, a single Executor contains the main executionloop and calls each Conductor to run its logic. Because the order in which callsmade on the Conductors is usually significant, the Executor must be edited byhand. Before calls to the new Conductor are added to the Executor, however, theExecutor’s unit tests are updated to ensure that all the Conductors are beingcalled properly. Within the Executor tests, CMock generated mocks are used to stand in for the actual Conductors.

Step 4 – Create Conductor Unit Tests & Production Code</p>

Starting from the system requirement currently being implemented, we next determine the individual functions necessary to satisfythat requirement. These are expressed as Conductor unit tests. Once the testsare created, production code to satisfy these tests is added to the Conductor.

The logic of the Conductor is tested against a mock Hardware and Model. Each function call made against the Hardware or Model must be addedin the Hardware or Model’s header file. CMock will automatically generate thecorresponding mocks from the Hardware and Model header files prior to running the unit tests.

Step 5 – Create Model & Hardware Unit Tests & Production Code

The Model and Hardware triad members are next implemented. Tests are implemented first; in the process, the interfaces for calls on the physical hardware and any interaction with other triad Models are revealed (interaction among multiple triads is discussed in depth in the MCH section).

Both the Hardware and Model will likely have initialization functions called by the Conductor. Functions in the Hardware and Model (already specified in the Conductor tests) may best be implemented by delegating responsibility to other helpers. In such cases, the module generation script is used to create these helper modules. Tests in the Model and Hardware will make calls against mocks of these helper functions.

The Hardware triad component, of course, makes calls to the physical hardware itself. Unit tests here generally involve modifying registers, calling the functions of the triad’s Hardware member, and then making assertions against the resulting register values. This can be complicated by read-only and write-only registers. Simulators often allow violations of these restrictions which may benefit or hinder test development.

Step 6 – Complete All Test and Production Programming

The tests and functional code of the Model, Conductor, Hardware, and helpers may require refactoring and subsequent re-running until the code is clean and all tests pass. When the unit tests are run, a Rake task will: call CMock to automatically generate Hardware and Model (and helper) mocks from header files, call Argent to insert Unity and CMock functions in appropriate files, use the dependency generation tool to create a list of files to be linked (including the unit tests, mocks, and Unity framework itself), run the compiler and linker, and then run the resulting test executable appropriately. Executing the test executable may require starting a simulator or flashing the target hardware and collecting test results from a serial port or other communication channel.

Once all the unit tests pass, the system can be compiled in non-test mode, loaded onto the hardware, and run against the system tests and hardware test fixture. Changes and fixes may be necessary to cause system tests to pass. Once tests pass, the new and modified files are checked into the source code repository; our automated build system will build and test the entire project and report results. Work on the next feature then begins anew.

Model Conductor Hardware Design Pattern & Conductor First

Design patterns are documented approaches to solving commonly occurring problems in software development. A multitude of patterns exist that address common situations in elegant and language-independent ways [12]. The Model Conductor Hardware pattern decouples functional logic from hardware for testing. With this decoupling, automated regression suites of integration and unit tests can be run on-chip, cross-compiled on a PC, or executed in a platform simulator. Working in this manner can even generate significant progress without target hardware.

Model Conductor Hardware Similarities to Model View Presenter

The basic MCH concepts and naming are modeled on the MVP design pattern. The Model of both patterns serves the same basic function – to model the system (e.g. mathematical equations, business logic, data access, etc.). The Hardware component of MCH is analogous to the View of MVP – both are lightweight abstractions that allow decoupling of system components and programming logic. The Conductor component of MCH serves a similar purpose to the Presenter of MVP – both define the order and composition of interactions within the triad upon an event occurring in either of the other triad components. Development in MCH starts with tests in the Conductor similar in fashion to the Present First [6,16] method of MVP development.

Model

The Model in MCH models the system (e.g. mathematical equations, control logic, etc.), holds state, and provides a programmatic interface to the portion of the system that exists outside a single MCH triad. The Model is only connected to the Conductor and has no direct reference to the Hardware.

Conductor

The Conductor in MCH conducts the flow of data between the Model and Hardware and is stateless. The Conductor acts when triggered by the Hardware or Model. It captures a system function in Model and Hardware interactions: setting state within the Model, querying the state contained by the Model, querying the state contained in the Hardware, moving data from the Model to the Hardware, moving data from the Hardware to the Model, and initiating the hardware functions encapsulated by the Hardware.

The Conductor was so named because of its role as a system director and because of its proximity to actual electrical components. Because of the correspondence to MVP, one could naturally assume that "Model Conductor Hardware" should be called "Model Hardware Conductor." The apparent incongruity is due to the importance we place on defining the triad interactions at the Conductor first. As the Conductor is central to the behavior of an MCH triad, we elected to name the design pattern to reflect this.

Hardware

The Hardware in MCH represents a thin layer around the physical hardware itself (e.g. ports, registers, etc.) and all its accompanying functions and state-based facilities. The Hardware notifies the Conductor of events by providing functions that check state in the hardware. State can be hardware registers or flags set by Interrupt Service Routines (ISRs) and other Hardware routines. The Hardware is only connected to the Conductor and has no direct reference to the Model.

Unit Testing with Model Conductor Hardware

With mocks constructed for each member of the MCH triad, clear testing possibilities become apparent (please see preceding sections for a discussion of mocks and interaction-based testing). The states and behavior within the Model are tested independently of hardware events and control logic. Each system requirement’s logic in the Conductor is tested with simulated events and states from the Hardware and Model. With mocks, even hardware register configuration code and ISRs can be tested. An MCH example follows in a later section of this paper.

Fig. 1. Relationships of a Conductor to its complementary MCH triad members and mocks. The depicted mocks stand in for the concrete members of the triad (linked in at build time) and allow testing of the logic within the Conductor. Mocks are automatically generated capture function parameters and simulate return values used in test assertions.

Model Conductor Hardware & Multi-Triad Systems

Multiple triads are necessary to compose an entire system and simplify testing. Multiple triads exist independently of one another. A central Executor contains the system’s main execution loop; it initializes and continually runs each triad by making calls on the Conductors. If there is communication necessary among triads in a multi-triad system, it will occur between Models. In a real-time system, individual triads can map well to tasks or threads.

Fig. 2. A system composed of multiple MCH triads. Inter-triad communication occurs among Models through direct calls or shared helpers. An external Executor runs each triad by making calls to service each Conductor.

Overhead & Development Cost

MCH adds little to no overhead (i.e. memory use or processor time) to a production embedded system. Of course, mocks and unit tests are not compiled into the final release system. Further, MCH is essentially naming, organization, and calling conventions; any overhead incurred by these conventions is easily optimized away by the compiler.

TDD shifts the time normally spent repaying technical debt [3] at project conclusion (and beyond) to the development phase with test-first practices. With this shift comes predictable development allowing decision makers to manage project risks with progress data. With TDD and CI clear savings are realized over the system’s lifetime in reduced bugs, reduced likelihood of recall, and ease of feature additions and modifications. New developers can work on the system and make changes with confidence because of test suites. Those tests, in fact, operate as "living" documentation of the system. Looking at the entire life cycle of a system, TDD and CI are considerably more cost effective than traditional or ad hoc methods of development.

Conclusion

Picture a desperate embedded systems engineer surrounded by wires, boards, and scopes, trying to track down a mysterious bug in a product that’s three weeks past its deadline. Now imagine a calm, productive developer over the course of many months regularly defining system behavior through tests and checking in code for the continuous integration server to build and test. Further, imagine a project manager confidently estimating and adjusting project completion goals and costs from measurable progress metrics. Regression testing, decoupled design, and disciplined development can all but eliminate heroic debugging efforts and costly defects released into production.

Acknowledgments

Authors wish to thank Matt Werner of Savant Automation for the opportunity to implement and originally prove out in a production system the ideas that inspired this paper. Authors also wish to thank their colleagues at Atomic Object for their many contributions to this work including but not limited to Presenter First, Systir, the dependency generation tool, and, of course, much proofreading and editing assistance.

References

[1] Kent Beck. Extreme Programming Explained. Reading, MA: Addison Wesley, 2000.

[2] Michael Karlesky, William Bereza, and Carl Erickson. "Effective Test Driven Development for Embedded Software." IEEE EIT2006. East Lansing, Michigan. May 2006.

[3] Technical Debt, http://www.c2.com/cgi/wiki?TechnicalDebt

[4] Nancy Van Schooenderwoert. "Embedded Agile: A Case Study in Numbers", http://www.ddj.com/dept/architect/193501924, November 6, 2006.

[5] Steve Freeman, Nat Pryce, Tim Mackinnon, Joe Walnes. "Mock Roles, not Objects." OOPSLA 2004. Vancouver, British Columbia.

[6] M. Alles, D. Crosby, C. Erickson, B. Harleton, M. Marsiglia, G. Pattison, C. Stienstra. "Presenter First: Organizing Complex GUI Applications for Test-Driven Development." Agile 2006. Minneapolis, MN. July 2006.

[7] System Testing in Ruby, http://atomicobject.com/pages/System+Testing+in+Ruby

[8] Ruby Programming Language, http://www.ruby-lang.org/en/

[9] Domain Specific Language, http://en.wikipedia.org/wiki/Domain_Specific_Language

[10] Measurement Computing Corp. miniLAB 1008,
http://www.measurementcomputing.com/cbicatalog/cbiproduct_new.asp?dept_id=412&pf_id=1522

[11] Argent, http://rubyforge.org/projects/argent/

[12] Kent Beck. "Simple Smalltalk Testing: With Patterns," 
http://www.xprogramming.com/testfram.htm.

[13] http://atomicobject.com/pages/Embedded+Software

[14] Martin Fowler. "Dependency Injection"
  http://www.martinfowler.com/articles/injection.html.

[15] http://rake.rubyforge.org/

[16] David Crosby and Carl Erickson. "Big, Complex, and Tested? Just Say ‘When’: Software Development Using Presenter First", Better Software Magazine. February 2007.

Page 2   Appendix    Back to the archive list