Mocking with JMockit

Chris Wash — Tue, 09 Jun 2009 05:09:58 +0000

Update: I cleaned up the example based on Rogerio’s comments.

Recently I stumbled onto JMockit and have been pretty impressed with the flexibility of the approach it takes.

Many mocking frameworks seem to take an elitist attitude toward testable code, not attempting to solve certain problems in favor of guiding one toward a more testable design. It appears JMockit is a response to this.

There’s no getting around the fact that some frameworks, especially legacy or proprietary third party modules, are not coded in such a way that it’s easy to write testable code against them. Common pain points include:

Pervasive use of statics
Lack of dependency injection mechanisms
Creating dependencies inline with the “new” keyword

All of these issues will pose problems when trying to double-out dependent code for testing purposes. One approach to solve these problems is to use a dynamic language and metaprogramming constructs to do this kind of doubling. Testing Java code with JRuby or Groovy has become more and more popular for this very reason.

But a lot of these will introduce a level of language abstraction between your test code and the code under test, and you’ll have to have developers maintaining a test suite sign on to learn the language being used.

In comes JMockit, which uses the instrumentation features provided with Java5 to perform a lot of the same magic tricks for you.

This also allows you to test things that weren’t otherwise possible (or are very difficult) and overall I think the programming model is much more in-tune with standard Java programming idioms. It doesn’t discriminate against you if you don’t use dependency injection, but will work with you if you do. Here’s a quick example that colleagues Andy Pemberton and Patrick Cox worked through with me trying out JMockit.

@RunWith(JMockit.class)
public class ControllerTest {

@Mocked
HttpServletRequest mockHttpServletRequest;
@Mocked
ServiceRemote mockServiceRemote;

@Test
public void test_execute_expectations() {

new Expectations() {
{ /* define in static block */
final Model m = new Model();
m.setId(12345l);

mockHttpServletRequest.getParameter("modelId"); returns("12345");
mockServiceRemote.getModel(12345l); returns(m);
mockHttpServletRequest.setAttribute("model", m);
}
};

/* simulate setter injection */
Controller c = new Controller();
c.setHttpServletRequest(mockHttpServletRequest);
c.setServiceRemote(mockServiceRemote);

/* call code under test */
c.execute();

/* strict mode will throw exceptions! */
}
}

You can get the whole project from my GitHub account.

Don’t Unit Test Anymore… No, Really!

Chris Wash — Wed, 18 Feb 2009 01:52:32 +0000

I just read Your Unit Tests Lie to You by Janusz Gorycki and I was going to leave a comment there, but thought it was more appropriate to expand my comments off into their own thing. For those that haven’t read the article, its basic premise is to grab hold of the nearest “test infected” reader and shake the warm and fuzzy out of them. It paints the short sightedness of many recent “unit testing” converts as living in a dream world where unit tests should replace formal testing. It follows with many sentiments I’ve read (and written about here) for a while now. It’s not that I disagree with what is being said in the article, or its tone for that matter; most of what is being said is spot on. Unit testing is definitely not a silver bullet. If you read my blog often, you no doubt get that. The article ends:

So please, don’t fire your QA department just yet. Their job is still important, even if you unit test.

So to Janusz, the fundamental problem here is a general ignorance of the purposes behind a unit test suite. I agree 100% that’s the primary factor behind his problem. What don’t we agree on? Semantics. But semantics are important! How far do we have to go for a true zen-understanding of this issue? Not far. Indulge me —

When is A Unit Test is not a Unit Test?

Here’s my thesis: you may use a unit testing framework, but what you write are developer tests. Even if they are technically unit tests, it is against everyone’s interest to call them this. Picky, useless distinction, you say? Hear me out.

There is a vast difference between the gamut of possible automated tests one could write and what is known colloquially as a unit test. A number of different kinds of automated tests are written against frameworks that are built on top of unit testing frameworks. That doesn’t make them unit tests. It doesn’t make sense to call them unit tests. A square is a rectangle, but does that make every rectangle a square? An automated acceptance or integration test is subject to a completely different set of problems (in areas such as specification, maintenance, complexity) than a unit test. In fact, about the only thing they share is their lifecycle and execution model, which many times has been retrofitted into the JUnit lifecycle and execution model.

“Unit Testing” and Linguistic Drift

I’ve recently seen a number of different incarnations (1, 2, 3) of the argument that we should eschew use of the term “best practice” because of the implications of its linguistic drift and general propensity of people to turn off their brains when just spoon-fed answers, not having to experience deriving the solution for themselves. Similarly, the popularity of unit testing frameworks and the sheer frequency with which the term has been used have, in a sense, set the idea of developer testing back considerably. Cedric Beust makes the point that in many cases we’ve confused [developer] testing terminology with JUnit terminology, and TestNG was in part a response to that. Here we oversimplify the problems we choose to bite-off and the goals we strive toward. They’re not realistic. Is it no wonder so many people fall flat when trying to adopt “unit testing”?

While there may be some overlap with the goals of validation and verification, most in the know consider the true benefits of “unit testing” to be a totally different animal altogether. We seem surprised to find the benefits of doing developer testing have little to do with what “testers” do. This dischord causes a lot of confusion, and has sparked a lot of articles. Some draw this conclusion, appropriately, that developer testing, while it fits a rigid definition of what testing is, shares little with what a typical “tester” is responsible for (true V&V). Quite often developers are the only ones doing any automation work, including this automated “developer” testing. Developers tend to do it for all kinds of different reasons, too. We’ll use a suite of automated tests to proceed without fear of integration errors. That adds value totally independent of validation and verification practices. If our suite catches regressions before we hand a single version off to testers, that saves both developers and testers time. I could go on and on, but the term “unit testing” conveys very little of these kinds of benefits to the development lifecycle, and as it turns out, causes a great deal of confusion.

The Right Usage

When we’re talking pure unit tests, that is, black-box testing components in pure isolation (which requires isolated dependencies), not in concert, as developers we can certainly find merit in this practice. But what we’ve learned is that this idea of unit testing, while quite beneficial as a development practice, shares very few goals with “testing” as we know it from a classical definition of the word (end-to-end V&V). Perhaps a few years ago this distinction was not that true, but we now know better. We know the danger in trying to bend unit tests into something they’re not.

My point is essentially this: knowing how to use a unit testing framework, a very simplistic construct on the surface, to do all of these wonderful things is certainly not something that just “falls out” of developing a unit of code – it’s not something we should expect developers to just deliver each iteration as part of their deliverable like it’s “done.” Effort invested to develop and maintain test code alongside the code under test is not free.

Only if you are persistent will you come to understand there sometimes is very little benefit in terms of “full blown”, classical validation and verification – the whole reason you set out on this crazy “unit testing” kick. You either stick with it, or write it off at this point. If you stick with it, well, I’ll save you the trouble of figuring this out for yourself; you learn it’s only another development practice, that needs to be balanced out with other sound practices. It’s hard work. Really hard.

Then why go to all the trouble? It’s my contention, and I assume most will agree, that developer testing is useful. It drives out better designs, code, and has the potential to thwart a caste of would-be regressions. It can stamp out integration errors. It can help you learn about a new framework or prototype a new feature. But that usefulness comes with a very real cost.

If we truly acknowledge that the type of practice we’ve talked about has merit as a development process, it makes very little sense to continue referring to the practice as “unit testing.” If testing modules of code, driving development with these tests, maintaining them alongside the code, and other automation activities are worthy practice for developers, we should consider the semantics of our vocational vocabulary. Let’s stop confusing each other. We’re not talking about delivering 100% code coverage, we’re not talking about replacing testers (unless you’ve got the huevos to go without them, anyway), and we’re surely not talking about writing or releasing bug-free software. It’s a lot easier to cut through a lot of the hype and confusion if we can all learn to say developer testing and understand it as such. Let’s put the term “unit test” to rest.

R.I.P. unit tests. Long live “developer tests!”

cwash into software » junit