Click here to view the complete list of archived articles

This article was originally published in the Spring 2006 issue of Methods & Tools

---

Continuous Integration: The Cornerstone of a Great Shop

Jared Richardson, http://www.jaredrichardson.net

I did a lot of lawn mowing when I was younger. My brother and I tried to make our summer money by asking real estate agents if we could mowthe lawns of their absentee clients. We'd usually land one realtor each year andthey'd give us enough business to keep us busy all summer. One of the things Ilearned is how hard it is to cut a straight line when you're mowing a wide yard.When I was in the middle of the yard, I felt like I was cutting a straight line,but then I'd get to the end of the row and look back to discover a crooked line.It always amazed me how something could seem so right and be so off course.

Timely Feedback

A software project can be a lot like mowing a yard. Even though we try to move in a straight line, and we think we are, later we lookback and are amazed at how far the project ran off course.

Whether mowing yards or building software, we need timely feedback to help keep us on track. Looking back at completed software projects, or lawns, shows you where you missed the mark, but it's usually too late for that project. We need feedback while we're still in the midst of the work. I never found a good way to get that feedback for my lawn mower, but I have found a guide for software projects. I use continuous integration systems to keep my projects on track.

Continuous Integration

Mike Clark calls this type of system a "virtual build monitor". This extra team member keeps an eye on your project and lets you know when things start getting off course. If you invest in a good automated test suite, you'll quickly catch all sorts of errors that traditionally pull good projects off course.

The more shops I get to observe, the more I'm seeing that continuous integration plays a vital role in keeping a shop on course. In fact, these days I'm telling people that I've learned one of the basic, fundamental principals to keep both you and your project on target.

Do you want to make your product a great one? Do you want to be the best developer you can be? Then make a solid continuous integration system a first-class member of your team and the cornerstone of your shop. A good CI system eliminates many of the problems that prevent you from working on the product, your career and your craft.

A continuous integration system does several things automatically.

• Monitors your source code
• Compiles after every change
• Tests your compiled code
• Notifies the developers of problems as soon as they occur

Figure 1. Continuous integration actions

As we move forward, keep an open mind and try to see where each step could’ve helped you in the last few months. Then, when we’re done, I’m going to point you to a Continuous Integration system that is trivial to install, easy to use, and open source to boot.

Let's look at what a continuous integration system is and why it helps so much.

The Steps of CI

Continuous integration systems all have a few common steps.

First, CI systems monitor your source code. The system usually watches your source code management system (CVS, Subversion, Perforce, ClearCase, Visual Source Safe, etc) but most systems can also monitor other resources, like file systems. This is how the software knows it's time for a build. Every time your code changes, the CI system checks out the latest version of your code.

Second, the software compiles your project. The system runs your existing build scripts by wrapping them in an Ant script. In this step, your CI software is requiring you to have a scripted build. If you builds are not robust or repeatable, your CI tool will expose this flaw. It will force you to have a clean build system.

Third, CI systems tests yours new build. The tests are created (or wrapped) in an Xunit framework (Junit, Nunit, HtmlUnit, jsUnit,etc), which means you have access to dozens of test frameworks that range fromunit testing to browser click through testing. When you set up a system to runtests, people are more likely to write the tests. They'll also contribute the tests they've been hiding on their own machines.

Lastly, your CI system will notify everyone of the results. The developers or testers who just changed the code will get email telling themhow long the build and test took, how many tests passed, how many failed, etc. Your system will also archive the results to a web page.

However, the publishing step is very configurable. You can publish in a variety of interesting ways beyond a standard web page. You canpublish to a custom web page, XML log, email, an instant messaging client, oreven a Lava Lamp. The publish step is an extremely flexible way of sharing your build results.

What's the big deal?

There are several key practices that continuous integration either requires or encourages. They are source code management, scripted buildsand test automation. Much of the benefit that comes from using a CI system actually comes from the foundational practices that a CI system requires.

Don’t get me wrong. CI adds plenty of benefit as well. It’s just that many day-to-day problems go away when you use these other practices first.

Code Management

One of the first things your CI system will do for you is make sure you've got your source code organized and (hopefully) into a sourcecode management system. After all, your CI software can't watch a code tree youcan't identify. The first practice CI encourages is good source code management.

This benefit will seem very elementary to many people, but I've seen shops that still use network drives and zip files. Quite a fewdevelopers still haven't discovered source code management.

Proper source code management doesn't take much time at all once you've learned how to use it. Like any good tool, you’ll save much moretime than you’ll spend learning to be effective with the tool.

You’ll save the time you normally spend reconciling code differences by hand, not to mention rewriting the work that careless coworkersoverwrite from time to time. Code collisions and lost work are common issues buta good source code system also merges your changes for you, maintains a history for each file, and more.

If you're not using a proper source code management system, I urge you to rethink you position. It's a huge time saver.

A Scripted Build

The second thing your CI system will require is a scripted build. Moving to this step requires a level of build automation. Fortunately,this is easy to add. There are many tools available, both commercial and opensource, that solve this problem for you. You still have to understand how tobuild your product, but these tools will keep from learning the differentcommand line options for javac or jar on different operating systems. Look at tools like Ant, Maven, and Rake.

Like good source code management, a scripted build provides many benefits.

For starters, your teammates aren't all busy building their own version of the build script. Everyone needs to build and developers, beingclever, will all find a slightly different way to solve the same problem. Whenyou have a single build script, everyone's building the same way. It's okay issomeone still wants to build differently (an IDE maybe?), but they need to have the ability to build the same way that everyone else does.

Don’t ignore the maintenance savings either. You'll eventually improve the build script, find a bug in it, or decide to make itfaster. With a single script, you do the work once time. When everyone has hisor her own build method, everyone solves the same problem repeatedly. What a waste of time!

When you build your code the same way, everyone gets the same version of the product. This means that the testers report problems in the sameversion of the program the developers run.

Without a canonical build script, you don't always get everyone on the same page. In fact, the customers, testers and developers oftenrun very different versions of the same product and then wonder why they can’treproduce the same issues. If you’ve had trouble reproducing your customer’sbugs, then start here. Is everyone running the same version of the product?

Page 2   Back to the archive list