cwash into software

Dwight D. Eisenhower said “In preparing for battle I have always found that plans are useless, but planning is indispensable.” After reading Jason Yip’s No matter how many times you say it, we still don’t need a QA on the team, I started thinking about how similar this is to the correlation between (software) quality and exploring what software quality is. In the spirit of Jason’s article, thinking about how you “get quality right” will have more positive results for the sake of quality than searching out defects. So the question stands, how do you get quality right? I’m not sure there’s a cut and dry answer, but I have a feeling it’s in the details. I know it sounds anecdotal, but quality isn’t easy. It requires a lot of effort applied constantly. It requires reward and incentive, not punishment and fear. It’s caring about small things, and shifting how your team thinks, communicates and works together. (continue reading…)

I’ve been doing a lot of reading lately and have been meaning to plug some of my favorite reads, and one of the things that I’ve been trying to read with an eye toward is for converting those that have been stuck on “behind-the-curve” projects to the new way of thinking and doing things. As such, and just in time for the weekend, I’ve compiled a list of my favorites with an eye to the “movers and shakers” that have driven innovation in the Enterprise Java world recently. Note that even though some of these links are specific to certain frameworks/technologies, there is a common thread throughout most of these that have a bigger focus on the best way to solve problems using Java 5/EE 5 constructs and concepts.

A book that should be on any Java developer’s shelf is Effective Java (2nd Edition). Make sure you get the second edition, for all of the Java 5 changes. This talk by Joshua should whet your appetite. (His talk on good API design is also very good.) You should also take a look at Bob Lee’s talk on Guice. Guice’s philosophy and focus on typesafety has spilled over into Web Beans (and newer versions of Seam) and its annotation based approach has heavily influenced newer Spring features. I especially like his points on type safety around the 11-minute mark. The walkthrough at the end of the talk from “old way” to “new way” is great for people who haven’t really gotten away from Factories, Delegates, and Service Locators yet.

The Guice video mentions that a common thread in modern Java frameworks is a focus on testability. Testing is a very important concept that nearly all new frameworks have embraced, but you still need know-how to be successful with testing. I’ve come across some great books recently in the automated (developer) testing arena. I’ve found the most thorough book on the topic out there is Lasse Koskela’s Test Driven. The ideas and topics discussed in the book are really language independent, and work equally well with other languages. To cut your teeth on TestNG, which the Seam documentation turned me onto, I recommend the TestNG developers’ book Next Generation Java Testing: TestNG and Advanced Concepts. Cedric Beust, also the author of EJBGen, and Hani Suleiman, author of the Bile Blog, produced a book that was a suprisingly good read. It’s packed with a lot of knowledge and some interesting takes on EE concepts. Newer versions of JUnit also include many of these features and concepts, so you’ll get the underpinnings of these concepts in this book. This video gives you a basis to TestNG’s approach.

As one of the growing number of developers who have an application in the Seam In Production page, there are a few Seam resources I have to share. Even if you don’t get into Seam, it should be on your radar because a lot of the core concepts around conversations have evolved into Web Beans, and are found other frameworks like Apache Orchestra and Spring WebFlow. Pete Muir did a Webcast that’s a great introduction to Seam 2.

Author of one of my favorite Seam blogs, Jacob Orshalick, has created a DZone RefCard for Seam 2.1 Core. It’s right off the presses, being published this week, but is a great quick-capsule review of what Seam can do for you, and nice to have on your desk if you’re writing Seam. If you need more in-depth descriptions of what’s on the RefCard, Jacob also coauthored the latest edition of the great Seam reference Seam Framework: Experience the Evolution of Java EE.
Dan Allen’s Seam in Action is another excellent resource. Of course, the awesome Seam Docs and Forums always come in handy. You should also RSS in.relation.to – the Hibernate/Seam developers’ group blog.

I know I’m leaving some great things out. What resources do you think someone who is used to J2EE should take a look at to get them “up to speed” with the state of the art?

If you’ve checked out my blogroll, you’ll notice I have a link to Basil Vandegriend’s “Professional Software Development” blog.  I like a lot of the articles he has written and to a large extent the subject matter I envision for this blog frequently crisscrosses with Basil’s.

While I was reading “The Reuse Trap In Software Design” I found myself thinking,”Me too! Me too!” like a giddy kid on the playground that’s found a new pal with a mutual interest in “pet snakes and/or tarantulas.”  As with many of the problems Basil writes about in his blog, I experienced the same problem, investigated it and found the same root causes, and came to many of the same conclusions as those outlined.  He describes the “reuse trap” as:

…[A] term I coined to describe the situation when one becomes stuck trying to design new functionality while simultaneously attempting to reuse existing code that needs some modifications.

He then goes on to describe why it’s a tough problem for n00bs and outlines a strategy he uses to get out of the trap–one that’s best described as a two-step process: copy-paste to reuse/refactoring to remove duplication.  I encourage you to read his article if you’re not familiar with the terms, or how this strategy can solve the problem described above.

My preferred technique for mitigating the problems involved isn’t copy-paste reuse/refactoring; it’s stubbing the calls to the code-to-be-reused (C2BR) that needs modification.  If I can fake out the behavior I need, it lets me focus on the nature of the dependency that exists between the two objects/components.  When I go to modify the C2BR, my inputs and outputs (or other consequences that occur as a result of a call to the C2BR) translate directly into a test that I can use to drive the changes I make to that code.  It also ensures that the coupling that exists between the objects is loose and legit.

The only thing to watch out for is that care must be taken to then remove the (small) duplication of the stubbed out inputs/outputs and actually wire the two pieces together when finished with the reused code.  I usually do that by throwing a TODO:WIRE marker in when I stub something.

I’ve found this is the approach I take more often, though I do take the copy-paste reuse/refactoring route from time to time.  This stub/rewire approach is a bit more advanced, but it’s how I envisioned solving the “reuse trap” when, as Basil puts it, “I became aware of it” and all that it entailed.

Do you ever come across this problem?  How do you get out of this “trap” or avoid being caught up in it in the first place?

In terms of the sheer number of moving parts, extravagance, and complexity, the build-tool landscape is one of bells and whistles that can be a little tough to stay on top of. There are a few important pieces of a robust and valuable build, no matter what tools you use to achieve it, and by breaking the build into modules or categories, we can make a little bit of sense out of the chaos.

In our vast build-tool landscape, there tend to be some zealous cults out there, entrenched deeply in their own camps and unwilling to venture out to explore what the rest of the landscape has to offer.   Thus, there have been some religious-like differences of opinion about which build tools are better and why.  Sadly you run across these types of biases all too often, which only pollutes the landscape and makes it more difficult to wrap your head the most basic questions underlying all tools:

• What does it accomplish?
• When is it (or isn’t it) right for the job?
• How do I use it effectively and efficiently?

With a pragmatic view, we can try to cut through a lot of the BS and figure out if a tool is worth its weight in:

1. dependencies/design complexity.
2. ease of use/interface complexity.
3. architectural decisions/compromises that it brings to the table.

Does the tool get us to the goal of a single button build/deploy?  If the answer is yes, I’ll welcome it with open arms.  Does it add me a form of feedback or some kind of statistic that could be a kind of gauge on a dashboard?  Then I may be interested in it, but it’s not nearly as vital.

Here’s an outline of what I typically look to include in a project’s build before I am satisfied:

• Compile – Obviously your build script would be for naught if you didn’t include this piece.  There’s also something to be said about some dynamic/interpreted languages (that run on the JVM) taking the hit on this step and producing compiled versions.  The argument is akin to compiling your JSPs back in the day.
• Unit test suite – Another one of those no-brainers!  You’ll obviously want to integrate your ability to run a single test or test suite from a build tool quickly and easily.
• Test reports – Test reports are an important way of visualizing the health of your project and are usually the first place you’ll go when you see something wrong.  A must have.
• Transitive dependency management – This is one of those sticky subjects that people like to argue about religiously.  Really the important thing to note here is that a crucial part of keeping your build clean and manageable is figuring out how to represent and resolve transitive dependencies, or second-level dependencies.  If your code directly depends on a library, odds are that library has its own run-time dependencies that need to be satisified in order for it to work properly.  Since managing the different versions of many different libraries can become tough when you start to look at different “stacks” of software, many people advocate having a tool help you resolve these dependencies so it’s not as daunting a task to get the entire stack to work again when you need to upgrade a single library.  Maven and Ivy solve this problem by introducing repositories that will house and dynamically resolve these versioning issues, in much the same manner as yum or apt-get does in the Linux world.  A lot of newbs will make the mistake of thinking this type of functionality will let them turn off their brain when it comes to think about their project’s dependencies.  They will be sorely mistaken!  The point of these to make resolving conflicts in transitive dependencies easier, they do not replace your brain, nor your obligation to know the dependencies in your project and how they fit together!
• Utility ant tasks – the daunting part of learning how to write good builds has a bit to do with how much you know about the ant task landscape.  Nearly all libraries have tools that plug into ant, but which ones do you really need to know about?  I’ve found a few are quite powerful, particularly replace, uptodate, cvstagdiff (and corresponding svn’s), xmltask and dbdeploy.  I think these tools should be looked at when you are trying to figure out how to accomplish something specific, and for that reason, they’re a bit more like “bells and whistles” than anything else.  The same goes for the next category, which arguably doesn’t even need to live in your build (but if you’re a team player, it’s a definite nice to have):
• Static analysis – There have been a barage of slick static-analysis tools that have entered the Java landscape recently.  A few are FindBugs, PMD, CheckStyle.  While most of these tools have plugins that integrate nicely into an IDE, it’s also helpful to have them plugged into your build, too.  Just to make sure everyone is following the rules.  A few others I’ve seen that focus on cyclomatic complexity, cohesion/coupling are Metrics and JDepend.

What build tools fit into your pimped out script?

Setting the Record Straight

A lot gets written about Continuous Integration, particularly on which is the best visual cue to let you know your build is broken or that a test is failing – lava lamps, Beta Brights, Ambient Orbs, and some even suggest traffic lights. But aside from this extraneous (at least to business) nerd-banter, a lot of what I find written about the actual topic of CI is fluffy, ivory tower, or pie-in-the-sky jibber-jabber that leaves out important parts of the big picture or confuses people more than it helps. In hopes of clearing up confusion on what exactly CI is and how it’s supposed to work, I’m ripping out a description that I wrote for a client proposal recently (so my apologizes for the dry-tone). I hope sheds some light on the true nature of CI, why it’s important and how to implement it from a birds-eye point of view.

Continuous Integration Dissected

Any large scale development project needs an automated, repeatable build process. Following best practices while developing a build process properly separates environment-specific configuration concerns from the codebase. This allows new environments to be created quickly and easily by simply overriding any environment-specific configuration values when first executing the build process. Whatever build tool is being used, builds should share a common, consistent process and interface. A consistent, repeatable build will know all of its dependencies and the goal is to be able to build any given module anywhere, independently, at any time.

Automated, repeatable build processes typically begin by obtaining dependencies (which can be specified using a dependency management tool) and a specific working-copy of the codebase (“checking out”) from a SCM system like CVS or Subversion. It is important to note that this codebase includes any code that is responsible for performing automated testing in addition to source code and configuration (and possibly other source-like artifacts).

Once the checkout has completed, the process will compile code and run automated unit test suites for each module in the system. At this point, all automated unit tests should pass, and custom development can begin. Any changes to code must be adequately covered by unit tests (either by changing existing tests or creating new ones), must fully compile without any errors and pass all automated unit test suites before being committed to the repository. The practice of always keeping the code committed to the SCM repository in this state (no compilation or unit test errors) is known as Continuous Integration and ensures that new development is safe to proceed at any point without fear of integration errors.

Subversion (and CVS) support concurrent development by following a Copy-Edit-Merge paradigm; any contention over files is usually caught when a developer tries to commit their changes and notices the underlying files have changed since they obtained their copy. In many cases, Subversion is capable of performing a merge automatically, if there was no contention over the same piece of a file. Sometimes, however, a manual merge will be required. Merging becomes more painful as the number of differences in the conflicting files increase. A good rule of thumb is that every developer should commit their changes at least daily.

Designating a single machine as a Continuous Integration (CI) environment provides many added benefits to a large scale development project. There are many operations which are good candidates to have run “continuously” but quite often are too expensive for developers to perform before every commit. Examples include executing automated in-browser system tests (which, if maintained over multiple releases, can serve as a “mini” regression test suite), performance tests/profiling, producing test metrics, generating documentation, etc. CI servers are an ideal place to schedule these processes to occur in an automated fashion.

You’ve done a thorough code review with your team and your code has come out with a few nice improvements. Good. You know that many a trustworthy source have speculated code reviews find as many as two times the number of defects that testing does. So you might suppose going to the trouble of creating automated tests, simply for your own devices, might be a bit overkill. As it turns out, it is worth it, and it’s going to save you a lot of time in the long run. Why is it so valuable?

(continue reading…)