Feb 1, 2010

Unit Testing in the Embedded World

You want to do WHAT?

I program both desktop based applications (C#) and embedded applications (C++). Their is a freight train of momentum behind the concept of unit testing (and TDD) in the desktop programming world. The embedded world is being carried forward by a Vespa. (NOTE: If you develop for the NetBurner, and just want to see how to use the library I built for unit testing your NB programs, scroll down and watch the videos.)
I went looking for a solution and took a gander at CppUnit, CxxTest, GoogleTest, the boost test library, and finally and most succesfully UnitTest++. In particular I was interested in a solution that would work with the NetBurner family of products and the Eclipse IDE (Galileo release).


If you find this article helpful check out my Eclipse Guided Tour course on Pluralsight. I’m currently working on part 2 which should be released in December of  2013.

Unit Testing for embedded projects presents some unique questions and problems. The first problem is: no matter what platform/IDE combination you use you aren’t very likely to find a plug and play solution. On the desktop world, once I installed Visual Studio 2008 Team Edition, doing unit testing was only a click away, and a lot of test scaffolding for modules could be generated automatically. CppUnit appeared at first to the most integrated choice for Eclipse and it provides what appears to be some nice automatic tools for generating tests. Unfortunately, it has to been compiled from the source and I could never get a clean compile using either Cygwin or MingGW, let alone getting it work with the NetBurner version of those libraries. I eventually found the barrier to entry too high and gave up. I then explored GoogleTest and CxxTest and found the same problem. I suspect these libraries all assume you will have a desktop OS at your disposal.
While it is conceivable that the unit tests could run under Windows (or an installed unix variant) the validity of such tests is at least somewhat questionable. The old embedded adage is “Test what you fly, fly what you test”. To me, this means at least running the unit tests on the embedded target itself. This means I’m willing to give up fancy output for a simple output of pass/fail messages.
I suppose some folks will think I’m insane for even bothering to attempt unit testing on an embedded system. Unlike a desktop I will always be limited in the amount of test coverage I can achieve. No matter how good the unit testing systems is, it’s never going to test that the hardware is doing the right thing and embedded programming always presents unique timing problems that just aren’t going to be unit testable. However, that doesn’t mean there are significant chunks of code that would benefit from unit testing and the regression testing ability implied therein.

Getting it to Work

Eventually I found UnitTest++. It appeared to very lightweight and ultimately proved adaptable to my environment. It uses MACROS to define tests, test suites and text fixtures.  The downloaded source code had some strange problems that I had to fix. For example a lot of class declarations didn’t end with a semi-colon and some namespace declarations did. I had to reverse that to make the GCC compiler happy. It wasn’t that difficult. There were also some modules for doing timing tests on both Windows and Posix systems. I just modified one of those to use the timing facilities available in the NetBurner. Then it was just a matter of including the typical NetBurner include paths as shown:
The framework also uses exceptions, support for exceptions is off by default in NetBurner projects so I also had to clear the checkbox to turn on support for exceptions as shown:
Once the library was built it was a simple matter to link it in to a new NetBurner device application and start using it. Of course since the test code will be an application this assumes you write your target test code as NetBurner device library. If you plan on working on more than one NetBurner project this is a good idea anyway. So ultimately I plan on ending up with one application that is capable of testing all my NetBurner libraries regardless of which projects they are used in. UnitTest++ supports the idea of test suites so I will leverage that to create a test application that can be configured and run on demand and may require certain hardware inputs for a test to pass. Using AJAX and the built-in web server should make creating a GUI Test Runner for the NetBurner fairly straight forward. Then of course I will also have the actual application per project. Any code in those projects that isn’t in the library will have to be tested the old fashioned way but that’s life in the embedded world.  It’s not a whole lot different that unit testing for any GUI based application. You typically don’t have unit tests to test the GUI, you just try to have as little code as possible in the GUI. In the NetBurner I will attempt to write as little code as possible outside of libraries.

How To Videos

Part 1 demonstrates how to set up the testing framework in Eclipse to work with the NetBurner.

Part 2 demonstrates a more realistic test.

Related Posts

Embedded Unit Testing: In the Trenches Part 1
Embedded Unit Testing: In the trenches Part 2

About Me

My photo
Tod Gentille (@todgentille) is now a Curriculum Director for Pluralsight. He's been programming professionally since well before you were born and was a software consultant for most of his career. He's also a father, husband, drummer, and windsurfer. He wants to be a guitar player but he just hasn't got the chops for it.