In Part 1, we covered the basic concept of a transaction, what the ACID properties are and how they specify transactional behavior, and why you might use transactions. This time, we’ll talk more specifically about what it’s like to use a transaction, and how the contracts it provides allow us to write data in a reliable fashion.

Programming With Transactional Systems

Commit and Rollback

Transactions either commit or rollback. Until a transaction is committed, it is assumed none of the work it has done is saved. Once a commit signal is issued by the application (either through an API call, or by a container), this work is actually performed by the transaction processing engine that’s being used. If for some reason the commit fails due to an error scenario, or some other runtime exception occurs, we assume that the integrity of the transaction has been compromised and the transaction rolls back, removing any work done up to that point. When a rollback occurs, the transaction must be resubmitted.

At its most basic, rollback is simply an undo mechanism, used to revert all changes back to the original state before the transaction began. To do this, we need to keep a copy of the original state before any changes, and if we need to rollback, we must reinstate the previous copy. Since there is always the possibility of multiple transactions running, concurrency control comes into play – we want to rollback to the last committed state, not just the state before beginning a single transaction. For example, consider this scenario: Alice and Bob both need to submit changes to a source code file under version control, and Alice and Bob must be able to turn in (commit) their changes at the same time in order to meet a deadline. The source code management system must ensure both transactions are consistent – that the data is recorded properly, but also make sure that the transactions are durable – that Alice’s changes (that were committed first) were not overwritten by Bob’s changes. If you had to implement this behavior that’s provided by the SCM system, how would you do it?

One way to think about the core of transaction processing engines is the Memento Pattern – as you perform actions within a unit of work that manipulate your application state, the transaction processor keeps track of sets of state changes and what the most recently committed state was. This way, when Bob goes to commit his work (after Alice), the SCM system can tell that the state of the resource he was working on has changed since he started his work, and can take steps to remedy the situation. This is a simplified view of what is going on: the steps to remedy Bob’s situation are not always straightforward. Also, providing this transactional behavior through network protocols, in a distributed environment, across different types of transactional resources, etc. all introduce complexity. But all complexity and bugs aside, the essence of transactions, that is, commit and rollback, should retain the simple semantics we outlined above.

Transactional API Code

We’ve talked a lot about transactions so far, but haven’t seen much in the way of code. To a large extent, we’d like to make transactions configurable, for many of the same reasons we tend to make other parts of our code configurable. But before going too deep into how to configure your code for transactions, let’s look at how you can control transactional APIs through code (“by hand”), first. Boilerplate code for working with a transactional resource often looks like this:

public class TestProcess {

	// ...

	public void process() {
		// open connection to resource
		// create transaction object
		// call begin on transaction object
		try {
			// perform interaction with resource
			// call commit on transaction object
		} catch (Exception e) {
			// call rollback on transaction object
		} finally {
			// call close on connection object
		}
	}
}

Problem: Transactional API code mixed in Application Code

Imagine writing code without using any transactions at all, that is, without any of the rollback semantics that we talked about above.  If the code is to work correctly in all scenarios, it needs to include a substantial amount of cleanup code.  Cleanup code and retry code, in general, is messy code. There are a lot of corner cases and expected failures that could occur, not to mention unexpected failures/error scenarios that need to be sorted out. The unassuming rookie programmer usually ignores these cases up front, and tackles adding error handling as he or she discovers the errors. This can make maintenance significantly more difficult because all business logic devolves into the arrow antipattern: becoming littered with if-else statements that inhibit readability and stifle the ability to refactor to a more suitable design.  While this is not a big deal in prototyping or non-critical scenarios, when we need to count on data integrity, it’s not an adequate approach.

Calling transactional APIs around your business logic can dramatically reduce the complexity of your cleanup code by providing rollback semantics.  In other words, when it is determined an error scenario has occurred and the transaction cannot be committed, we rollback, or restore the application to a previously consistent state.  This often means “undo-ing” pending work, but is often implemented by queuing up work to be done and only making those changes when commit is called.   Still, code that interacts directly with a transaction manager or monitor API leaves something to be desired in terms of maintainability.  Applications constrained to be implemented by working with Transactional APIs tend to make them inherently procedural, which can be more difficult to test and maintain. Because your business logic is surrounded by another API, often this makes code harder to read as well. For these reasons, we favor using declarative transaction management in cases where your application demands implementing more than a trivial amount of transaction code.

Declarative Transaction Management

Transaction Attributes

Transaction attributes allow us to declare how the container should handle interfacing with transaction APIs on our behalf, eliminating a lot of the boilerplate transaction API code and messy cleanup code altogether, allowing it to be specified and live in configuration metadata. I asked a question on Stackoverflow that challenged the community to come up with a good metaphor for how these worked. I would encourage you to read the link to the OpenEJB description, the examples/use cases/metaphors there, and add your own.

Transaction Attributes in Programming Models

System.Transactions in .NET, and EJB3 in JEE5+ provide transaction attributes that allow for defining the way your application code should interact with the underlying transactional resource. See the references section below for more information on each of these models.

Advanced Transaction Concepts and Terminology

Distrubuted (XA) Transactions

When committing work across two or more different transactional resources, we use a “distributed” transaction, also known as an XA transaction. (XA stands for the industry-standard XA interface defined by the X/Open group.) In Java, support for distributed transactions is provided by libraries that implement the Java Transaction API (JTA) – most often you’ll find these libraries included in the runtime of JavaEE application servers. N.B. that use of JTA does not necessarily imply that you are using distributed transactions – it simply provides a space for you to define your own “global” transactions, that may include XA and/or “local” transactions.

XA transactions are much more complicated because in order for the ACID properties of all resources to be guaranteed, each system must implement a “Two-Phase Commit” protocol that ensures transactional consistency across all resources. Support for this protocol is often included in more robust versions of drivers or connectors and used only when needed, as they introduce complexity and overhead that degrades performance. It’s beyond the scope of this article to describe this protocol, but the reader is encouraged to investigate how two-phase commit works if it is in use in his or her environment.

Local transactions do not support the XA protocol – the transactional semantics are only guaranteed to work against a single resource. Let’s look at an example that will contrast the two models. Consider an application that must write some information to a database and also enqueue a message to a message queue (for example, enqueuing a message that triggers a confirmation e-mail to be sent). If we are using local transactions, we must ensure that the message is not enqueued after the data has been committed to the database. This sounds like an easy enough request to accommodate, and may to lead you to use only local transactions. If there is a problem writing the data to the database, no e-mail will ever be sent, because the global transaction (containing the database updates and message enqueue) will rollback before it ever tries to send the email. But what if the enqueue of the message fails? Without using distributed transactions, there is no way to “uncommit” the database changes. There are certainly other strategies to mitigate this problem without using a distributed transaction, but they involve writing cleanup and retry code. If both the writing of the data to the database and the enqueue of the message are part of a global, distributed transaction, then any problem enqueuing the message will force the entire transaction to rollback, in which case the client will need to resubmit the entire transaction again. Obviously there are pros and cons to using either approach, but this example should give you a taste of when and why you would choose to use a distributed transaction. Distributed transactions are useful when data integrity is paramount, but are more difficult to configure and become challenging as the ability to scale and handle highly concurrent transactions increases.

Compensating Transactions

When no rollback semantic is available to us, we often find ourselves in a situation where we’d like to issue a transaction that will “undo” a specific transaction that was already previously committed. This is the idea behind compensating transactions. They’re to transactions what anti-particles are to elementary particles in physics; one will “cancel” or “equal out” the other, the net overall effect being to compensating the state of the system. To use compensating transactions, for each transaction type you define the steps that will perform the opposite work to reverse a committed transaction. Compensating transactions are often used in lieu of distributed transactions, but they can also be used to undo an entire distributed transaction that has been committed. They’re especially useful for transactions that are very long-lived, for instance, as part of a long business process, but can suffer from the same data integrity issues that they are used to address – by their very nature they are themselves “cleanup/retry” code. The idea of a compensation transaction is used heavily in the BPM/service orchestration space, where several remote service calls must be glued together into an overall unit of work. In these cases, compensation transactions can be vital to keeping the orchestration logic in a well-known state, especially when you don’t have control over the transaction demarcation or a way to enlist in a transaction for systems that you are integrating with. We won’t talk much more about them, as the mechanisms that provide this behavior are not standardized, and could be the topic of a series of blog articles all by themselves.

Up Next

• Transaction Isolation
• Concurrency Control

References

• http://openejb.apache.org/3.0/transaction-annotations.html
• http://www.simple-talk.com/dotnet/.net-framework/.net-2.0-transaction-model/
• http://en.wikipedia.org/wiki/Two-phase_commit_protocol

More Related Content

• No Related Post

Preface

This work came out of an lack of consistent, cohesive documentation for beginners on transactions. Similar material does exist, but much of it suffers from either being hard to find, scattered and spotty, or not written at a basic enough level where prerequisite knowledge is not required in order to understand the material covered. Over the course of this series of articles, we’ll talk about what concepts you need to know in order to effectively and correctly make use of transactions in your applications. This series of articles is written as a set of core concepts to understanding the basics of transactions, and is intended for junior to intermediate level developers, but my hope is that others will benefit from this series as well. As always, feedback is greatly appreciated.

What is a transaction?

Etymology

The word transaction comes from the Latin word transactionem meaning “an agreement, accomplishment,” which itself comes from the past participle of the verb transigere, transactus, meaning “drove or carried through.”

Transactions defined

The word’s definition is typically related to business or economics, meaning a single business deal, or an exchange of goods between two parties. Most introductions to transactions use an example of debiting one bank account and crediting another. When we remove funds from one account, we need to make sure that they are deposited into the other. If something fails (sufficient funds do not exist in the account being debited, or the account being credited has been closed) then both accounts need to be returned to their original state before the exchange began. This is actually a very good example, but how does that apply to software?

Let’s back up for a second. If we focus on what’s going on, we begin to see there are two distinct activities in play:

1. Grouping a set of multiple changes together that all have been applied successfully, or, in the event of a single failure, that all need to be undone.
2. Managing (or coordinating) the execution of these changes, when their interactions might interfere with one another.

Examples of Transactional Systems

Like many other terms in the software lexicon such as “class”, “object”, “system” and “code”, the definition of the word “transaction” suffers a bit from the word’s inherent over-abstraction; it is widely applicable to a great many circumstances and situations. But once understood, is easy to spot scenarios where the pattern is used and applied. Transactions are important in software when updating shared resources, like a cache, database, or message queue, but also come into play when reading – especially when access must be coordinated across many separate but concurrent workers reading shared data. We’ll talk more about this in depth later. Some examples of systems that heavily utilize software transactions are:

• eCommerce (Shopping-Cart, Billing, Banking) Applications
• Relational Databases
• Caching Systems
• Messaging Systems (Message-Oriented Middleware)
• Source Code Management Systems

As you can see, not only are these fundamental building blocks of business software, but also modern distributed systems. Transactions are prevalent in the frameworks and tools we use to build, tune, and scale the software we write.

Specifying Transactions

ACID properties

A technological definition of the word transaction involves specification of qualities (or “properties”) that must be enforced in order for the transaction to be processed successfully. These properties, the ACID properties as they are known, grow out of our business concept of a transaction (an exchange between two parties). They exist as conventions that specify or constrain the software that’s responsible for processing transactions. Enforcing all four of these properties guarantees the transaction is reliable and therefore viable candidate for successful completion/processing.

• Atomic – When we say transactions need to be atomic, we don’t mean that they need to be small, though often they are. What we mean is that you can’t subdivide a transaction any further – the group of changes must be viewed as a single set. When applying that set of changes, it’s “all or nothing” – that is, each and every individual change processes successfully, or else the entire group is undone completely. There are no “partial” transactions; we can’t subdivide them further.
• Consistent – If a system successfully processes a transaction, then all resources involved in the transaction must be consistent afterwords – whether the transaction processed successfully or not. For example, if one account is debited and another is credited, the balance of the debited account must be equal to the difference of the original balance and the price of the transaction (and vice-versa for the credited account). Transactions that do not complete successfully should not have any side-effects to either party involved in the transaction.
• Independent – Transactions should not interfere with one another, even though they may involve shared resources. For example, one transaction should not be able to see the work another transaction is currently performing until that transaction has completed. They should be independent or isolated. Typically, transaction isolation becomes an issue as transactions being processed become larger: that is, the longer they live or the more resources that they enlist. Isolation also has performance implications as the amount of load or concurrency increases. Isolation mechanisms expose either locking or multi-version concurrency control (MVCC, “diff-ing” or delta-management). We will discuss these later when we talk about concurrency control and isolation.
• Durable – Systems processing transactions must record successful transactions so that a history of the transactions processed is not lost. Relevant actions taken are logged for auditing purposes. This audit trail allows the state of the transactional system to be recreated at any point in time if there is a system failure. If the information cannot be recorded in this way, we can not say we have a durable transaction, because we won’t be able to recreate the state of the system if a failure occurs.

In a real world business transaction, these conventions are likely enforced by human beings involved in the transaction. If one of these properties were violated during your checkout at a store in the mall, then either you or the clerk would be pretty upset and likely to call in relevant authorities to sort out the matter. For example, if you bought a shirt but were refused a receipt for the transaction, you would most likely be upset and demand a refund, as the receipt is confirmation of the transaction’s durability.

Why Should I Bother Using Transactions?

As you can see, transactions also provide us with some guarantees about what will happen in the event that something goes wrong during the interactions between shared resources. These guarantees are very useful to us as software designers and developers, but their benefits may not be intuitively obvious until we consider what our jobs would be like without them. Transactions give us with a safety net of assumptions; when something goes wrong, we can lean on the transaction’s guarantee to rollback, or automatically revert all changes from the start of the unit of work. Imagine if we didn’t have this guarantee: we would be forced to write code to cleanup the mess we’ve created, and not only is this code hard to maintain, it’s hard to write and get correct the first time. With transactions, we don’t have to worry about writing extra code to “cleanup” the mess we’ve created, and can instead focus on what the code we’re writing is actually supposed to do: the business logic. When something goes wrong, we can safely assume certain scenarios are out of the question by using transactions. In many circumstances, transactions are key to writing code that is functionally correct, and most of the time make the code cleaner, more robust and maintainable (especially when considering the alternative of coding cleanup logic by hand).

Transactions are fundamentally all about coordinating access to shared resources. A term that can be used interchangeably with the word “transaction” is unit of work. Transactions are important to software systems because they’re how we manage interactions with the different systems involved that have access or expose shared resources to clients.

Typically when we work with transactional systems, as application developers we:

• Write code that explicitly utilizes a transactional API, like JDBC, that will contain the semantics for how to handle the processing of the transaction, or…
• Configure a transaction management mechanism, like an EJB container, that can do the processing and coordination work on our behalf based on conventions.

The next article in the series will provide examples of both approaches. Either way, the end result provides a way to keep each unit of work’s interactions separate from each other, and guarantee that all changes in the set are applied together or undone completely.

Up Next: Writing Data Reliably

• Programming With Transactional Systems
• Declarative Transaction Management

Further Reading

• http://www.etymonline.com/index.php?term=transaction
• http://en.wikipedia.org/wiki/ACID

More Related Content

• January 7, 2010 -- Eliminate Branching (IF Statements) to Produce Better Code (0)
• July 24, 2009 -- The Elements of Reusable Code (0)
• July 29, 2009 -- What I’d Tell Myself About Design If I Were Just Beginning (5)
• June 9, 2009 -- Mocking with JMockit (5)
• June 3, 2009 -- What is Hamcrest? (0)
• April 15, 2009 -- OSGi-ggity-Giggity (4)
• January 31, 2009 -- In response to Stackoverflow #38/”Quality Doesn’t Matter That Much” — Jeff and Joel (3)
• January 13, 2009 -- On Software Quality (8)
• November 28, 2008 -- Must Haves/References For Modern Java EE Developers (1)
• November 19, 2008 -- Java 6 and Maven 2.0.9 on Leopard (7)

The following scenario plays out a lot when I pair up with people, so I thought I’d distill it down and write it up for future reference. I think teaching lessons through scenarios could make for an interesting series of articles, and I’m sure others have equally interesting experiences to share.

I’ve tried to condense this down to just the steps vital to understanding the underlying problem (though this is certainly not the only way to get there) and introduce some terminology (providing links, even if deferring to google) along the way. I hope these help out aspiring or new developers.

Here’s a scenario: you find yourself doing X all over the place, (where X is nearly anything) and you’re just plain tired of writing the code to do it. You first reluctantly copy and paste it into new places, and twiddle what you need to twiddle to get it to work. You soon realize this is a pain because it’s error prone – you can mess things up using copy and paste. You fancy yourself the pragmatic type and decide to write a IDE template that will fill in the code for you instead of having to write it out every time.

(Aside: This is but one of many kinds of code generation).

If you are using code generation, you stopped asking “why” too soon. -mahvne

This helps with the problem of introducing errors, but you soon you realize this code is everywhere, cluttering up other code. Not only that, when the structure of this code changes a little bit, you need to go and change it everywhere it was copied and pasted, or generated using your template.

(Aside: It’s a violation of the DRY principle.)

Now you decide the best course of action is to try get this code to a common place that’s visible across every part of your project. You also have to go find every place you copied and pasted or used your template and replace them with calls to your method that use generic parameters.

(Aside: You’ve just created a static utility method.)

You make up your mind that in the future, it will be much easier to do this before the code gets copied and pasted that many times, and vow to follow the three-strike rule of refactoring.

(Aside: And use automated extract method refactoring and move method refactoring to do this).

The lesson to take away is this: any line of code you introduce creates change points and increases complexity. We need to make sure the lines of code allow into our code base are well-factored/designed/tested and worthy of our ongoing maintenance efforts. Really – try to break your code before it leaves your hands. Fix it, or get rid of it. While this might seem cynical at first, you’ll understand it’s just preventative maintenance, or programming deliberately (not programming by coincidence).

More Related Content

• No Related Post

In a recent tech talk, I watched Miško Hevery propose an interesting challenge to his audience: start a toy project and try to write the code with no if-else or switch blocks at all. None at all?  Before thinking about how to do this, why would you want to do it in the first place?  On the surface, it may seem to the unassuming a bit counter-intuitive.  Comparison-based branching is at the heart of programming, and the concept of an if-else or switch block is almost universal, existing in most every programming language ever devised.  What’s Hevery’s agenda with this exercise – and what can possibly be gained from trying to write a program without branching entirely?

Branching Considered Harmful?

In 1968, E.W. Diskjstra, famed programmer and author of the shortest-path algorithm, wrote an article to be published in the Communications of the ACM entitled “A Case Against the GOTO Statement” that was later renamed “Goto Statement Considered Harmful” by the editor.  The phrase has grown into a meme of its own in the CS community, denoting a practice that has fallen from grace and become widely discouraged.

I bring this up not to point out its cultural signficance, but the core of his argument:

The go to statement as it stands is just too primitive[;] it is too much an invitation to make a mess of one’s program.

Essentially, Hevery is making this very point for the if-else statement, 40 years later, but for very similar reasons.  If-else and switch statements easily account for some of the ugliest code that I have seen (and written).  Overuse of if-else blocks can lead to the arrow antipattern, a scenario where conditional branches are so deeply nested they become unmaintainable.  Too often developers fall into this trap, described aptly by another Dijsktra quote:

The prisoner falls in love with his chains…

Hevery’s point is that code without if (or switch) statements is:

• Easier to read and understand
• Easier to test
• Easier to maintain, extend, etc.

But how do we write code without branching constructs?   If we are talking about fixing the arrow antipattern, you may know a time when you avoided this antipattern by using separation of concerns, for example, the MVC pattern instead of a monolithic servlet.

Hevery’s Suggestions

Hevery goes on to explain how good OO design can eliminate the need to explicitly code the most egregious uses of branching, by instead favoring dependency injection and polymorphism.  But this technique is only really useful where we can create methods and objects that are cohesive and loosely coupled.

With polymorphism, you dispatch into a method whose binding is not known at compile-time but determined at runtime.

This is the weapon of choice when we are working on code that is highly procedural and there is a bad OO-design smell.  But it is far from a one-size-fits-all solution.  If we were to transform every branch in our program by instead using polymorphism, it would be a tangle of incohesive objects that would likely be harder to maintain than the original procedural code.  Luckily, there are other techniques that are more appropriate in certain circumstances.

Hevery continues: certainly, primitive comparisons are going to be the most difficult to eliminate, but shouldn’t happen very often.  So we reluctantly allow these to stick around.  Another class of if statements exist to check against nulls.  Hevery explains how null pointer dereference checks quickly become pandemic, occurring every time a member of an object is referenced.  This should largely be a non-issue, and is only a problem because code is allowing nulls to be returned.  The semantics/method contract of most methods can easily be changed to prohibit the return of nulls.  (This is only really difficult when recursion is used.)  Developers, as it turns out, don’t have to return null values.  This is particularly true with collections, and warranted a section in Joshua Bloch’s Effective Java.  Instead return an empty collection can be returned, or you can implement the null object pattern (e.g., a no-op logger).  Hevery finally rallies for ditching using return codes in favor of throwing relevant exceptions.

Note: I encourage you to watch the talk, as he continues talking about how specifically you can use inheritance and polymorphism to remove branching. The rest of this article will talk about other strategies to remove unnecessary branches.

Remove Copy-Pasted Loops And Conditionals: Introduce Enumerable Operations

Many languages contain an enumerable construct that includes “higher order” looping operations that can eliminate the need for many specialized cases that are usually solved by a foreach loop with an embedded conditional check of some sort.  This type of construct can be traced back to Smalltalk, and exists in most modern dynamic languages such as Ruby, Groovy, Python, and provided by a number of Javascript libraries.  Getting this to work in Java is a bit of a challenge because there is no mechanism for closures (yet) but some partial solutions do exist (they lack the full power of true closures, but overusing true closures can break down OO-style constructs).  For further information, take a look at lambdaj, Adrian Kuhn’s Pimp My Foreach and Google Collections’ Iterators.

Here is an example of the types of things this style of programming can fix.

List<Integer> numbers = Arrays.asList(0, 1, 2, 3, 4, 5, 6, 7, 8, 9);
List<Integer> oddNumbers = new ArrayList<Integer>();

for (Integer check : numbers) {
    if (check % 2 != 0) {
      oddNumbers.add(check);
    }
}

In the case of the Google Collection, they call their enumberable construct a “Iterable” and the closure construct a “Predicates,” we’ll be able to replace the above code with this:

Predicate<Integer> oddIntegerPredicate = new Predicate<Integer>() {
    public boolean apply(Integer input) {
        return input % 2 != 0;
    }
};

Iterable<Integer> oddNumbers = Iterables.filter(numbers, oddIntegerPredicate);

LambdaJ, and Kuhn’s solution work in a similar fashion.  While syntactically not as pretty as the dynamic languages, these constructs are functionally equivalent and arguably no more painful than the original code.  In many cases of deeply nested for-if-for-ifs, it significantly reduces the cyclomatic complexity).  This style of programming, once everyone on the team has become familiar with it, can reduce often copy-and-pasted looping code to one liners, removing some really ugly duplication and the potential for those hard-to-spot copy and paste errors.

Lookout for Train Wrecks: Obey the Law (of Demeter – and other SOLID Principles).

Steve Freeman and Nat Pryce describe “train wreck” code in their new book Growing Object Oriented Software Guided By Tests as chained “getter” method calls, linked together like train cars.  The authors call them “wrecks” because the code is essentially an egregious violation of the Law of Demeter.  One of Nat’s examples:

employee.getJobDescription().getResponsibilities().add(new ManagementResponsibility(employee.getDepartment()));

The problem is that this style of programming destroys all encapsulation and information hiding concepts in code.  Train wrecks can become pandemic as well because instead of using an object-oriented style of programming that passes messages between objects to perform work, we essentially allow objects to know too much about their collaborators and details.  Another way this has been said is “talk only to your friends” or “ask, don’t tell.”  A client object asks other objects what their state is, and uses an if statement to perform an action, instead of telling the collaborating object what it wants done, and letting that object figure it out.  Instead of the train wreck, we should extract the behavior to a method that allows us to tell our collaborator what we want done:

employee.promoteToManager();

Complex conditional “train wrecks” can often be extracted in the same way.  Quite often these types of objects are really just procedural code masquerading as a class, and will be in violation of other SOLID-principles such as the Single Responsibility principle.  Making this code more “SOLID” will make it easier to test and maintain.

The Bigger Picture: Well Designed, Testable Code

Remember, this exercise in design (by removing ugly branching) is not just for aesthetics.  There is a very legit reason to try these techniques: writing testable code.  Hevery’s talk (and his other talks) are essentially about how to write code that is testable.  Freeman and Pryce’s book is about the very close relationship between testability and OO-design.

If you like these ideas, you might be interested to try the TDD As If You Meant It exercise as well.  This is an exercise that will show you how to listen to test code to drive adding features that are driven out by tests into your production codebase by implementing them first in the test class.  This is the ultimate way to ensure that your production code contains only what it needs and nothing more, and that code is adequately covered by tests!  Remember, the more code you write into production code, the more you’re going to need to maintain it, and the more skeptical you should be about how readable and well designed it is.  These techniques and exercises may seem silly on the surface, but if you explore them I am confident they will help you write better code!

More Related Content

• July 8, 2010 -- Transactions, Part 1 (0)
• July 24, 2009 -- The Elements of Reusable Code (0)
• July 29, 2009 -- What I’d Tell Myself About Design If I Were Just Beginning (5)
• June 9, 2009 -- Mocking with JMockit (5)
• June 3, 2009 -- What is Hamcrest? (0)
• April 15, 2009 -- OSGi-ggity-Giggity (4)
• January 31, 2009 -- In response to Stackoverflow #38/”Quality Doesn’t Matter That Much” — Jeff and Joel (3)
• January 13, 2009 -- On Software Quality (8)
• November 28, 2008 -- Must Haves/References For Modern Java EE Developers (1)
• November 19, 2008 -- Java 6 and Maven 2.0.9 on Leopard (7)

I recently read Scott Leberknight‘s “Can Java Be Saved?” with interest and started typing up a response.  As Scott’s article was a bit lengthy, so became the comments I had in response.  In excess of 1000 characters, which JRoller told me was spam when I finally tried to post it.  I’m still never surprised to see that happen to my writing.  Anyway, I’m probably just an oaf, but I didn’t see Scott’s e-mail listed anywhere on his site (probably another spam-deterrent choice) so I thought since I spent more than a few minutes on it, it’d be appropriate enough to post here and link it.  Sorry but I don’t have the energy to evolve this into a more formal post, perhaps I will at a later date.

A Newardian post deserves a Newardian reply!  I really do like your suggestions for language additions. But those kind of additions are going to be happening in the context of the JCP, and that’s its own can of worms. I think you can directly attribute people’s despise for Java, or maybe more aptly, their enthusiasm for new languages, to the difficulty involved in making language changes to Java. Regardless, there is still freshness to the innovation in the Java world. People are just looking for it in the wrong place.  Certainly, the virtual machine ecosystem is alive and well, as you point out.  But in terms of innovation within Java, it’s happening in the programming model, not the language.  Some of Spring, Guice, and CDI’s enhancements to the programming model are novel ways of evolving Java given the restrictive “all things to everyone”, bureaucratic nature of language changes at this stage of the game.

Here are a few examples of programming model innovation. The native property syntax reminds me of another recent blog entry by Gavin King.  Sure, it’s a bit more verbose, and lacks the punch of a native language feature. But is it that much different? Also, do I really need type inference? I see it as easier to read and more powerful to leave the variable declaration and remove the duplication by nixing the call to the constructor, and annotating a producer method. It also gets rid of the new keyword, which was your next item on the list. Again, this is a programming model enhancement that’s independently evolved and proven itself.

I think there is still a lot of room for evolution, and a lot of work being done. As you point out, the 800 lb gorilla seems to be continued by-in and enthusiasm from vendors – but most if not all of the real innovation that is occurring in the community is through open source software. It’s interesting how you have a rift in the open source space, in how, e.g. the Spring and the Seam teams have approached developing open source, and the willingness to reintroduce innovations to the JCP.  Is the JCP to be used as a springboard for adoption, or a roundtable to resolve pain points everyone is feeling?  Its days as a think-tank for ivory tower specs are gone.  The fate of Java is not yet sealed, but I think in terms of the language, we’re only really going to see problems that everyone feels are painful get pushed through diplomatically.  But that’s not to say that there are programming model enhancements that can be pushed for adoption.  Vendors seem to be more open to evolving the programming model than the language at this point, if someone is willing to baby a spec through the process.

I certainly welcome any of your thoughts on this. Thanks for the interesting article.

More Related Content

• November 28, 2008 -- Must Haves/References For Modern Java EE Developers (1)

The competent programmer is fully aware of the strictly limited size of his (or her) own skull; therefore he (or she) approaches the programming task in full humility, and among other things he (or she) avoids clever tricks like the plague.
–E.W. Dijkstra

Designers are something of an enigma to most people; however, the fruits of their labor are largely understood, well used and appreciated. Design is often considered to be a purely aesthetic matter: a matter of taste, of style, of opinion; we know a good design when it when we see it.  But many times we can’t quite put our thumb on why it’s good design, or what makes it well-thought out, or pleasing. Maybe some of us can articulate what it is we like about design, but few connect the dots enough to notice recurring qualities in our favorite designs or think about underlying truths beneath those qualities.  Even fewer still have the experience to know when to apply these qualities, within the proper context to produce a good design.

Programmers are equally if not more engimatic to general society.  Perhaps on the surface for many different reasons.  But, coincidentally developers are designing all the time; could this play a part, perhaps?  Is this why we refer to the most experienced of our profession as “architects?”  Aren’t we, by way of programming systems and wiring them together, inherently striving for greatness as designers?

Especially when you’re inexperienced, you’re very much focused on learning what it is about a design that you find compelling, agreeable, or pleasing.  We value a good design so much that we spend a lot of time studying, talking about and debating what makes a good design.  As the narrator (Jack?) in Fight Club admits he had become “a slave to the IKEA nesting instinct,” so too, I find myself in willing servitude of the design pattern instinct at times.  More often nowadays I stop and catch myself, and I’d like to think I’m not the only one with this premonition.  Too quick are we to whip out our design patterns books (especially right after we’ve finished them) and, as the first order of business, go to town solving problems we don’t have.

Few products today need to ship in shrink wrapped boxes as they once traditionally had to.  The connectedness of the Internet, coupled with all of the modern advances in hardware have fundamentally changed the way we look at developing software.  “Fast, cheap, good – pick any two” still applies, but fast is faster, cheap is a lot cheaper, and good encompasses much more today than ever before.  Software development still lags behind.

The unwise developer makes all design decisions up front, for fear of getting the solution wrong.  The enlightened developer creates the closest thing to a working solution now, for fear of getting the solution wrong.  The former produces a solution that fits his view of the problem.  The later continually adapts his solution to encompass others’ view of the problem.  Which is more valuable?  The unwise developer wants to be done, and will sacrifice alienating some users.  The enlightened developer wants to be correct, and will compromise stability of the design.  Why do this? Because inactivity leads to software rot, and we have techniques for dealing with changing designs.  What good are our big up-front design skills when the requirements change?

Like Darwin encouraged us to do with our views of biology, we’ve ditched an up-front design model for an evolutionary one.  There is increasingly a focus on software as an evolutionary process everywhere, and it’s phrased in many ways: continuous production (Cal Henderson’s explanation the best); the perpetual beta; “no such thing as done“, agility – openness and responsiveness to change.  The risk of bloated software has lead us down a path to focus on and seek to create small, modular applications that create “business” value immediately, can be chained together but never grow overly complex.  Our design decisions still exist, but they are on a much smaller scale, and with much more concrete drivers and concerns factored in.  And we understand that we’re fundamentally in error when we make them out-of-context.

Others have thoroughly dealt with this topic in much greater detail than I can or care to.  Neal Ford has written about what Evolutionary Architecture and Emergent Design look like.  Martin Fowler has asked, “Is Design Dead?” and answered his own question.  I would encourage you to read their thoughts on the matter, as my own conclusions are largely in line with theirs.

That being said, there are some tools that you should look to add to your collection as you go.  Any craftsman must learn to master his or her tools.  Just having the tool in your toolbox does not necessarily make you a good designer; because they must be used properly, in the proper context.  Like Clint Eastwood’s character explains of his toolkit in Gran Turino: each tool has a job, its own context and realm of applicability.  Also, resist the urge to apply them every chance you get.  Much of good design is about restraint.  Try to see them applied correctly, or incorrectly, and talk with others about it.  Keep a conversation going; don’t allow your codebase to become stagnant.  Continually question, strike the right balance and good designs will emerge and bad designs will improve. Your methodology may have you delivering in sprints, but good design is a marathon.

Here are the basic tools.  There is a lot of material out there on these – easy to Google or find on Wikipedia.  I am not going to offer any links or speculation, because I think these things should continually be discussed.  You should be constantly evolving your own opinions and knowledge of them.  They must be used, discussed and understood in order to be mastered.

Overarching Computing Principles – Reduce Complexity

• KISS – Keep it Simple, Stupid.
• DRY – Don’t Repeat Yourself.
• YAGNI – You Ain’t Gonna Need It.
• SOC – Separation of Concerns.

Basic OO Principles

• Inheritence
• Coupling/Cohesion (dependency)
• Encapsulation
• Modularity
• Abstraction
• Polymorphism

SOLID (Advanced) OO Principles

• Single Responsibility Principle
• Open/Closed Principle
• Liskov Substitution Principle
• Interface Segregation Principle
• Dependency Inversion Principle

Emergent Design Techniques

• Test Driven Development
• Continuous Integration
• Refactoring
• Project Automation
• Continuous Production

To borrow a great closing quote from Brian Liles,

Do not seek to follow in the footsteps of the wise. Seek what they sought.
–Matsuo Basho

More Related Content

• July 24, 2009 -- The Elements of Reusable Code (0)
• July 8, 2010 -- Transactions, Part 1 (0)
• January 7, 2010 -- Eliminate Branching (IF Statements) to Produce Better Code (0)
• June 9, 2009 -- Mocking with JMockit (5)
• June 3, 2009 -- What is Hamcrest? (0)
• April 15, 2009 -- OSGi-ggity-Giggity (4)
• December 1, 2008 -- 24ways (1)
• November 28, 2008 -- Must Haves/References For Modern Java EE Developers (1)
• November 19, 2008 -- Java 6 and Maven 2.0.9 on Leopard (7)
• August 2, 2008 -- How I Escape the “Reuse Trap” (0)

Back in April, there was an interesting article quoting Ron Jeffries et al on InfoQ looking at code reuse from an agile perspective. The conversation steered toward explaining reuse as a concern that is very expensive, looking at it from a top-down, “enterprise”-wide lens.

But is code reuse a black and white issue? My contention is there are varying degrees of reuse that are often neglected on a microscale within an even moderately-sized project. If the code you produce isn’t going to be reused, is it even going to be looked at? These are similar problems with similar solutions.

As far as producing reusable code, I believe there is a medium of full-fledged, framework or API degree of reuse and also a “just enough” degree of reuse. The article touched on a similar concept:

Adam Sroka called this ‘emergent reuse’ and noted that this seemed more efficient than the ‘design for reuse’ approach. A poster named Tim explains it to his business people this way: “The first time you have to pay for it to be coded, the second time you have to pay for it to be reusable, the third time it’s free.”

A higher degree of reuse requires more of an investment in these principles and practices. What’s more, I think the article fails to draw a distinction between the roles (or competing concerns) involved in reuse: not only should we be worried about reusing code we produce, but often we are presented with a very real problem that wasn’t talked about in the article which is how we consume code that we choose to reuse.

A project is itself a consumer of its own code, and many times you’ll find a codebase is not even internally consistent in how it reuses code within itself. Quite often consumer-reuse happens in the form of depending on third party libraries, open source frameworks or APIs. In this sense, reuse becomes an issue of code quality, or more aptly, code cleanliness.  It’s hard to come up with solutions to big problems by just thinking about an ideal solution, and even if you are able to do it, odds are your solution will probably be rather academic.  It’s easier to make progress on large-scale problems by breaking them down, and making a habit of looking for patterns and ways to combine, compose and chain smaller solutions.  Like we do with Agile problem solving approaches.  Can’t we do that for the hairy reuse problem?

Decomposing the Problem

Code reuse is a tough problem to nail down, because it’s a high-level view composed of a whole host of smaller concerns. With a nod to separation of concerns, I’ve tried to decompose these competing factors into their own categories and make some suggestions of things to look at if you’d like to have a higher degree of reuse within your team or code base.

Communication Concerns

Communication Saturation

Last night I listened to an Agile Richmond talk by Jeff Sutherland espousing the ideal of communication saturation.  Jeff was talking about Scrum, and wanted to draw a strong correlation between this saturation and “hyper-productive” teams.  The key, Jeff said, was the ability of the group to recognize what is the next most important thing to work on and come to a collective decision about it.  This resonated with my experience with agile.  Our group was able to defer stories where we saw reuse was going to be apparent in the future, and this not only did this save us time in the long run, but it made the problem of “discovery” and consumption a no brainer.  Everyone on the team knew a component we delivered one release was to be reused in future releases, even if we weren’t involved in developing it.  This, coulped with the idea of collective code ownership, opens all the right communication channels for a software development project in my opinion.

Value cross training

Don’t be afraid to ask for help or give it if you can provide help. Educate yourself and teach others. You’ll find that successful teams do this naturally through many different ways. If it seems tough to approach individuals, a better course of action may be to schedule a weekly development meeting to discuss hard problems and solutions to those problems. Record topics that come up regularly. An informal retrospective after a release or milestone can go a long way to correcting the course of the ship.

Design Concerns

Cross cutting concerns

A whole class of reuse problems can be boiled down to reinventing the wheel instead of solving core business problems.  If you’re solving a technical problem, is it a cross-cutting concern?  If you can identify a problem that has been solved in more than one way, it’s a good idea to pull the team together and talk about a good way to solve it moving forward.

Refactor mercilessly to remove duplication

Refactoring to remove duplication is a skill that takes practice and diligence to do well. But it’s often a quick win to spot and remove duplication; Don’t Repeat Yourself will go a long way to improving the quality of the code your team produces by coercing the design of the code into something more fundamentally sound and amenable to larger scale refactoring later.

Reuse problems are often a manifestation of not having “everything in its right place.” If you always know where to go to get a hammer, then the problem is reduced to that hammer being there when you go to get it.  The bigger problem is (rightfully so, now) knowing when you need that a hammer.  This is not necessarily going to solve the problem of discoverability or consumption for you (communication is needed for that), but it will help with organization tremendously.  Whether or not you achieve an enterprise degree of reuse may be a horse of a different color and have a lot to do with communication, but in the microscale refactoring is the weapon of choice.

Names Matter

If you are able to refactor using composed methods, naming becomes a lot easier. Each method will aim to solve one single “atomic” concern, and larger methods will become more readable and logical as they get shorter and more maintainable.

Technical Concerns

Know the language and learn standard APIs

Unit testing can help you learn a new API fairly quickly and document some hard-earned pieces of knowledge going forward. A handy construct that is available in many languages is a shell interpreter like IRB or BeanShell, which can be invaluable in helping to pick up the syntax and semantics of a language. Wikipedia has a list of interactive programming language shells.

Modularize your code

Modularization is important. Few languages support full modularization OOTB. At the base level, you need to be able to encapsulate your code and also package or namespace it. This is all you really get OOTB in most languages. Ideally, full modularization would provide you with the ability to bundle together code and provide a mechanism to have the code installed or uninstalled dynamically at runtime – in order to achieve this you often have to specify dependencies between modules and declare a version for your bundle. (OSGi aims to do this in Java, and is definitely worth a look).

Modularization is important to code reuse because it presents a barrier to someone else using your code. If I want to use a particular component or method that exists in your library, that means I am fundamentally absorbing the lowest-level dependencies that you have. If your project is not properly modularized, I may have to eat all of your dependencies. The larger your component is, the more complexity any consumer must absorb to reuse your code. Modularization should help with this problem.

SOA was supposed to have solved this problem, right?  At least a subset of reuse problems – iteroperability.  We’ve found it’s a lot harder in practice than it is in theory.  You have to version services, and use some registry to look up and discover a version.  Not to mention the governance issues involved.  This is why I’m so excited about OSGi.  This might be the way to take your microscale reuse to the enterprise, and the model (in practice, and arguably theory) is a good deal simpler than SOA, too.

Documentation Matters?

If you are writing an API or framework, you can’t argue that documentation will be incredibly important to the success of your project. Note, I’m not talking necessarily here about Java-doc style-documentation; rather, I’m talking about providing a would be reuser of your code with enough support to make them successful. Sometimes that means concise, well written and up-to-date API documentation. Quite often this will come in the form of a user or developer’s guide with a more narrative style or structure. FAQs or troubleshooting guides are often very important tools.

Reuse is something that is far easier to say than to do. Doing it requires both good design and very good documentation. Even when we see good design, which is still infrequently, we don’t see the components reused without good documentation.

-D.L. Parnas, Software Aging. Proceedings of the 16th International Conference on Software Engineering, 1994

Going Forward

We all know waste is a thief.  Rework is wasteful.  Solving the problem of reuse on a grand scale may seem intractible, but I think it’s just another problem, and arguably not even the hardest one we have to contend with in software development.  If we can break the reuse problem down from its mighty enterprise-wide incarnation into smaller, more digestable reuse problems, it becomes a lot easier to make progress.  And maybe one day we’ll find a way to get to that enterprise-wide reuse that may seem too good to be true at the moment.

More Related Content

• August 2, 2008 -- How I Escape the “Reuse Trap” (0)
• July 8, 2010 -- Transactions, Part 1 (0)
• January 7, 2010 -- Eliminate Branching (IF Statements) to Produce Better Code (0)
• July 29, 2009 -- What I’d Tell Myself About Design If I Were Just Beginning (5)
• April 15, 2009 -- OSGi-ggity-Giggity (4)
• June 9, 2009 -- Mocking with JMockit (5)
• June 3, 2009 -- What is Hamcrest? (0)
• January 31, 2009 -- In response to Stackoverflow #38/”Quality Doesn’t Matter That Much” — Jeff and Joel (3)
• January 13, 2009 -- On Software Quality (8)
• December 1, 2008 -- 24ways (1)

Update: I cleaned up the example based on Rogerio’s comments.

Recently I stumbled onto JMockit and have been pretty impressed with the flexibility of the approach it takes.

Many mocking frameworks seem to take an elitist attitude toward testable code, not attempting to solve certain problems in favor of guiding one toward a more testable design. It appears JMockit is a response to this.

There’s no getting around the fact that some frameworks, especially legacy or proprietary third party modules, are not coded in such a way that it’s easy to write testable code against them. Common pain points include:

• Pervasive use of statics
• Lack of dependency injection mechanisms
• Creating dependencies inline with the “new” keyword

All of these issues will pose problems when trying to double-out dependent code for testing purposes. One approach to solve these problems is to use a dynamic language and metaprogramming constructs to do this kind of doubling. Testing Java code with JRuby or Groovy has become more and more popular for this very reason.

But a lot of these will introduce a level of language abstraction between your test code and the code under test, and you’ll have to have developers maintaining a test suite sign on to learn the language being used.

In comes JMockit, which uses the instrumentation features provided with Java5 to perform a lot of the same magic tricks for you.

This also allows you to test things that weren’t otherwise possible (or are very difficult) and overall I think the programming model is much more in-tune with standard Java programming idioms. It doesn’t discriminate against you if you don’t use dependency injection, but will work with you if you do. Here’s a quick example that colleagues Andy Pemberton and Patrick Cox worked through with me trying out JMockit.

You can get the whole project from my GitHub account.

More Related Content

• February 17, 2009 -- Don’t Unit Test Anymore… No, Really! (6)
• June 3, 2009 -- What is Hamcrest? (0)
• January 31, 2009 -- In response to Stackoverflow #38/”Quality Doesn’t Matter That Much” — Jeff and Joel (3)
• November 28, 2008 -- Must Haves/References For Modern Java EE Developers (1)
• July 8, 2010 -- Transactions, Part 1 (0)
• January 7, 2010 -- Eliminate Branching (IF Statements) to Produce Better Code (0)
• July 29, 2009 -- What I’d Tell Myself About Design If I Were Just Beginning (5)
• July 24, 2009 -- The Elements of Reusable Code (0)
• April 15, 2009 -- OSGi-ggity-Giggity (4)
• November 19, 2008 -- Java 6 and Maven 2.0.9 on Leopard (7)

As of JUnit 4.4, if you happen to peek into the distributed JAR you’ll notice something a little off: in addition to the org.junit.* packages there is this a funny-looking org.hamcrest.* package sticking out like a sore thumb. You may have seen other projects pick up a dependency on Hamcrest lately as well, and I bet you’re wondering what it is. Let’s get to the bottom of it.

Introduction

The “I’m Feeling Lucky” Google search for the term hamcrest currently takes you to the project’s Google Code page that states Hamcrest:

Provides a library of matcher objects (also known as constraints or predicates) allowing ‘match’ rules to be defined declaratively, to be used in other frameworks.

That’s a pretty good description if you already know what a Matcher is… but it doesn’t explain much for a beginner. So what does Hamcrest really do for you? 

It's just like... It's just like... A mini... REGEX!

A good way to think of it is that Hamcrest is to objects what regular expressions are to text.
Hamcrest provides you with a set of methods that effectively define a DSL to do pattern-matching on objects.

What can you use it for? Well, like regular expressions, the possibilities are endless. Some really novel uses have come about recently, but the most prominent use has been in the area of testing – hence the JUnit dependency.

A Brief History

Hamcrest evolved out of the library JMock, being used to write specialized constraints on what you expected to happen to a Mock object when standing in for a real implementation. A more fluent assertion syntax arose from these constraints which allowed you to chain together main constraint calls under a single assertThat method. Later JMock’s author, Joe Walnes, refactored the constraint API out into its own library called Hamcrest as people became interested in using it outside of a testing framework for all kinds of different things. He also started calling the constraints “matchers” though they can go by both names or, in some circles, “predicates.” Other testing frameworks and even JUnit, notorious for not requiring any dependencies and staying conceptually small, have picked it up.

All those old friendly org.junit.Assert.* static methods are still around in JUnit 4.4+. They’re useful for certain things, but by and large they’ve fallen out of style. The new kid on the block, JUnit’s assertThat method allows you to pass a Matcher in to create an assertion that you would otherwise use a specialized-assert method. Why is assertThat so useful? A few different reasons. Not only does it provide a more generic way to specify assertions, it also alows the framework to know specifically what you’re trying to assert. That allows the framework to generate failure messages for you.

Fail message… win?

It’s always been a good idea to use the overridden version of assertX method that also takes a String for a failure message, for example:

If we passed in an object representing a childhood version of Blackbeard, say, (before he took to the high-seas) then we’d expect to see our failure message:

It’s always been a good idea to provide these messages. But there’s no getting around the fact that they’re repetitive, tedious, and redundant.

Using Hamcrest matchers alongside assertThat makes JUnit smart enough to generate meaningful failure messages for you: writing your assertion with Hamcrest matchers provides the framework with enough contextual information to generate meaningful failure messages on your behalf. No need to maintain an extra arbitrary String to make sense of your failed assertions:

This may not look like much of an improvement, but it also doesn’t read any worse. We can ditch the extra String parameter as well. When this fails we’ll get a pretty cool failure message for free:

Note: I’m using JUnit 4.6 – in order to get this syntax to work you need to do some static imports:

It looks like there’s still work being done to make this DSL read more fluently in JUnit; something like:

Other Cool Stuff

Getting Reused

TestNG, JMock and many other testing frameworks will certainly benefit from what Hamcrest offers, but we’re starting to see this being used in many other places as well. I’ll be interested to see how well matchers work together or are maintained seeing that they’re relatively easy to create. Hopefully frameworks will contribute useful matchers back to the project.

Chaining

Because the matchers are implemented as a DSL using the method chaining idiom, they’re extremely easy to combine and extend.

Matchers For Regular Expressions

To bring the blog post full circle, I found this earlier tonight: the Hamcrest Text Patterns project. Its goal is to have you write Hamcrest-style regular expressions using matchers which will produce much more readable regular expression code. For example, have a look at this test that I found in the project:

By providing a friendly name for each capture (user, host, scheme, email) you can provide meaningful messages programmatically about what doesn’t match up when something doesn’t match.

Conclusion

I hope you choose to take a closer look at Hamcrest because it brings a big bang-for-the-buck and has the potential to guide us to more human-friendly, readable code.

Further Reading

• Have a look at Joe Walnes’ blog for a great introduction to using assertThat as well as other creative uses of Hamcrest matchers.
• Also take a look at Nat Pryce’s blog for more info on Hamcrest and Testing in general.

More Related Content

• June 9, 2009 -- Mocking with JMockit (5)
• April 15, 2009 -- OSGi-ggity-Giggity (4)
• November 19, 2008 -- Java 6 and Maven 2.0.9 on Leopard (7)
• July 8, 2010 -- Transactions, Part 1 (0)
• January 7, 2010 -- Eliminate Branching (IF Statements) to Produce Better Code (0)
• July 29, 2009 -- What I’d Tell Myself About Design If I Were Just Beginning (5)
• July 24, 2009 -- The Elements of Reusable Code (0)
• February 17, 2009 -- Don’t Unit Test Anymore… No, Really! (6)
• November 28, 2008 -- Must Haves/References For Modern Java EE Developers (1)
• September 10, 2008 -- Axis2 client using ADB – runtime dependencies (6)

Update: A great article on OSGi popped up on Javalobby today. Check it out.

I haven’t yet written any thoughts about OSGi but it’s something that’s increasingly found its way on to my radar over the past year and a half or so. I’ve been doing a little bit of reading and research on it lately, (a quick introduction can be found via Adrian Colyer’s talks on InfoQ about it). Needless to say it’s got me excited. Really excited – to the point where I’m catching myself geeking out uncontrollably like Quagmire from Family Guy. What’s got me all giggity? Let’s take a step back first.

With Java there has always been a focus on modularity, but it’s come up in different intermediate forms. Any typical Java developer can go on for hours about the importance for layering and properly isolating, decoupling and packaging components and subsystems. Java has the idea of a package, and the idea of JARs came about fairly early on in the process. But what became difficult was that often each layer had to have its own supporting infrastructure in place to be built independently, and someone needed to know how all of those JAR dependencies mapped out at build time and runtime. This was a necessary evil so that we could have individually deployable modules or subsystems. Once you got there, it was all worth it.

The problem was that while you may have been diligent and rigorous in your approach to applying how your projects were laid out and built, quite often the framework space solved this problem over and over again in many different ways. Going to upgrade a library was not an easy task, because you had to typically upgrade it across each of your layers, and what’s more, there was no consistent approach to manage transitive dependencies amongst the modules each library used. Library selection and dependency management was the work of a greybeard within your project and it took a while to change a library or upgrade. Not to mention a near certainty that you’d need to execute your entire suite of regression tests to make sure nothing broke.

After seeing many Java frameworks being cajoled into a Maven build in order to manage builds and transitive dependencies, there has been increased awareness on the part of the vendor (be it commercial or open source) space on modularity. But we’ve only focused on the problem of solving modularity from a build-centric view. In other words, “How can I package and manage these build artifacts in such a way that consumers can easily integrate them into their project in a consistent and automated way?” This was the pie-in-the-sky idea behind Maven as I see it, and they pretty much made it happen. You’d be hard pressed to find a framework today that doesn’t support it in some fashion.

But there’s a whole separate piece of modularity that takes place at runtime. In the context of my historical explanation of modularity within Java, you can look at OSGi’s ability to manage dependencies dynamically at runtime (via a header in your JAR MANIFEST file) as a “modularity win” and icing on the cake. But OSGi indeed goes farther than that. OSGi gives you the ability to install/uninstall, start and stop bundles dynamically (think Eclipse plugins) via its lifecycle, and that just plain gives me the “giggities.” To account for truly dynamic modularity OSGi also gives you requires and provides semantics (also declared via headers) akin to the requires and provides semantics you see in dynamic languages like Ruby and some Javascript packaging/dependency management frameworks. While this may seem like its strength is only in making a clear separation between APIs you use and export, its real strength is that this allows you to truly isolate your modules at runtime from trickery that can (and has) happened on a classloader level. “That couldn’t possibly work on an App server,” you say? I was skeptical, too, but not so fast…

If you’re the type that keeps up with the JCP space, you probably know there is a JSR “trinity” (equally as mysterious as its holy counterpart to the outside viewer) to provide a more robust modularization mechanism in varying degrees. That “quagmire” withstanding, OSGi has been mounting a considerable lead as the modularization system of choice in the JavaSE and EE/App server space. (It was already a formidable player in the embedded/mobile space, where it evolved from.) Nearly all major App server vendors have integrated the OSGi model into their core, which means they’ve probably cleaned up any classloader nastiness they’ve resorted to in the past. And now you’re seeing some of the leading frameworks like Spring and Web Beans/Seam 3 (and subsequently have signed up to publish themselves as OSGi bundles.) Not to mention the success that Eclipse has seen since refactoring itself to work off its OSGi implementation, Equinox.

If OSGi can continue to make inroads in the framework space, I think its prospects to the “in the trenches” Java developer are very exciting indeed. Expect me to babble on more about this in the future…

More Related Content

• July 24, 2009 -- The Elements of Reusable Code (0)
• June 3, 2009 -- What is Hamcrest? (0)
• November 28, 2008 -- Must Haves/References For Modern Java EE Developers (1)
• November 19, 2008 -- Java 6 and Maven 2.0.9 on Leopard (7)
• March 19, 2008 -- New Wave Logging (0)
• July 8, 2010 -- Transactions, Part 1 (0)
• January 7, 2010 -- Eliminate Branching (IF Statements) to Produce Better Code (0)
• July 29, 2009 -- What I’d Tell Myself About Design If I Were Just Beginning (5)
• June 9, 2009 -- Mocking with JMockit (5)
• January 31, 2009 -- In response to Stackoverflow #38/”Quality Doesn’t Matter That Much” — Jeff and Joel (3)