cwash into software

Setting the Record Straight

A lot gets written about Continuous Integration, particularly on which is the best visual cue to let you know your build is broken or that a test is failing – lava lamps, Beta Brights, Ambient Orbs, and some even suggest traffic lights. But aside from this extraneous (at least to business) nerd-banter, a lot of what I find written about the actual topic of CI is fluffy, ivory tower, or pie-in-the-sky jibber-jabber that leaves out important parts of the big picture or confuses people more than it helps. In hopes of clearing up confusion on what exactly CI is and how it’s supposed to work, I’m ripping out a description that I wrote for a client proposal recently (so my apologizes for the dry-tone). I hope sheds some light on the true nature of CI, why it’s important and how to implement it from a birds-eye point of view.

Continuous Integration Dissected

Any large scale development project needs an automated, repeatable build process. Following best practices while developing a build process properly separates environment-specific configuration concerns from the codebase. This allows new environments to be created quickly and easily by simply overriding any environment-specific configuration values when first executing the build process. Whatever build tool is being used, builds should share a common, consistent process and interface. A consistent, repeatable build will know all of its dependencies and the goal is to be able to build any given module anywhere, independently, at any time.

Automated, repeatable build processes typically begin by obtaining dependencies (which can be specified using a dependency management tool) and a specific working-copy of the codebase (”checking out”) from a SCM system like CVS or Subversion. It is important to note that this codebase includes any code that is responsible for performing automated testing in addition to source code and configuration (and possibly other source-like artifacts).

Once the checkout has completed, the process will compile code and run automated unit test suites for each module in the system. At this point, all automated unit tests should pass, and custom development can begin. Any changes to code must be adequately covered by unit tests (either by changing existing tests or creating new ones), must fully compile without any errors and pass all automated unit test suites before being committed to the repository. The practice of always keeping the code committed to the SCM repository in this state (no compilation or unit test errors) is known as Continuous Integration and ensures that new development is safe to proceed at any point without fear of integration errors.

Subversion (and CVS) support concurrent development by following a Copy-Edit-Merge paradigm; any contention over files is usually caught when a developer tries to commit their changes and notices the underlying files have changed since they obtained their copy. In many cases, Subversion is capable of performing a merge automatically, if there was no contention over the same piece of a file. Sometimes, however, a manual merge will be required. Merging becomes more painful as the number of differences in the conflicting files increase. A good rule of thumb is that every developer should commit their changes at least daily.

Designating a single machine as a Continuous Integration (CI) environment provides many added benefits to a large scale development project. There are many operations which are good candidates to have run “continuously” but quite often are too expensive for developers to perform before every commit. Examples include executing automated in-browser system tests (which, if maintained over multiple releases, can serve as a “mini” regression test suite), performance tests/profiling, producing test metrics, generating documentation, etc. CI servers are an ideal place to schedule these processes to occur in an automated fashion.