Agile Testing at SkillsMatter

SkillsMatter, a training site here in London, have started a new series of free evening training sessions on agile testing, many of which they convert into video podcasts. The latest session video looks particularly intriguing, with reports from the field on testing activities in real projects. There’s a summary of what’s covered that looks promising. Maybe we should have a showing in the office?

Bad uses of Template Method

Template Method is one of the original GoF patterns – an obvious way to use inheritance to share functionality, but I’d hazard it’s more risk than it’s worth. Here’s an stripped-down example (in pseudocode):

  class A {
    util() { return true }
    algo() {
      if (util()) { /* do some stuff */ }
      else { /* do different stuff */ }

  class B extends A {
    util() { return false }

Parent class A has a function algo() which uses another function util() to tune its behaviour. Derived class B provides an alternative version of util(), so an instance of class B calling algo() uses the implementation of algo() in class A but uses the util() provided in class B.

There’s some obvious problems in the way I’ve written this: algo() has two modes of behaviour but you can only access one of them through instances of class A. To get the other branch you must call algo() via an instance of class B. If you unit test thoroughly you’ll discover that you can’t test all of class A from its unit tests – that’s a bad smell. The function util() tells class A that it’s actually being run by an instance of class B – that smells too.

But maybe it’s not bad enough to discourage. After all, you can see that something extra is happening to get to the else branch of that if statement.

Try a slightly more complex case:

  class C {
    boolean state = true
    setState(x) { state = x }
    util() { return state }
    algo() {
      if (util()) { /* do some stuff */ }
      else { /* do different stuff */ }

  driver(C c) {

Class C looks quite complete in itself, and your unit tests of class C could reach every line. Nothing up my sleeves!

But perhaps you want to change the behaviour of algo() for a particular case; one very common solution is to introduce something like this:

  class D extends C {
    util(x) { return true }

Use driver() with an instance of class D and the code (which you didn’t touch at all) suddenly stops going down the expected path, even though you can see the explicit call to setState() in driver() telling to take the else branch!

Debugging this type of situation is very difficult. There’s nothing in class C or in driver() that vaguely hints at this changeable behaviour. It might not even occur to you that there is a class D. You’d probably start debugging by just staring despairingly at the code for class C.

Of course there are a bunch of strategies for solving this sort of puzzle, getting the flexibility and reuse but keeping the behaviour change out in the open. So try to recognise the pattern, know it by name, and develop an instinct for its risks.

Being Singly Responsible

The theme of our latest kata was the Single Responsibility Principle [PDF]. The Single Responsibility Principle says that every class should have just one responsibility, or alternatively just one reason to change.

For an easy example, consider a Screen class that represents a computer monitor. You might give it members like driver, connectionType (VGA or DVI), windowsDisplayed, and draw(Rectangle r). It’s pretty easy to see that these fall into two categories – some are attributes of a Device that the operating system talks to, and others are attributes of a Display that programs want to draw on. A remove() method might be in either category – it might remove the device, or remove the objects on the display! We can find plenty of examples of this kind of confusion in some of our larger and less tame domain classes.

Like the Screen example, physical objects, on which we often model our software objects, have multiple responsibilites. In fact a Screen may play more roles – accountants see it as a depreciable capital item and the IT department see it as a trackable asset, for example. Where those responsibilities appear closely linked, it’s very tempting to combine them in a single object, and hard then to see that they should be separated. This makes the principle easy to state, but difficult to apply.

For example, consider an Employee. Two things we may want to do are to evaluate the employee and to pay the employee a salary. These are closely linked in our minds – we expect a positive result in one to cause a positive change in the other! But there is no reason to link the code, though it seems natural; we will have much more flexibility if changing the evaluation mechanism doesn’t mean having to change the payroll system also. This promotes safety, too – the less we can touch a sensitive system like payroll, the better.

In some code we’re writing to automate our release process, there was a ReleaseService responsible for

  • assigning IDs to release requests,
  • requesting files from the ProductStore where our application artifacts live,
  • releasing those files, and
  • reporting on status.

That’s a lot for one Groovy class to handle! After applying the SRP, we have an IdService that assigns IDs (which incidentally makes it easy to verify that the assignment is threadsafe), a ProductStoreService that handles interaction with the ProductStore, and other services that copy and release .war files. The ReleaseService is responsible for orchestrating these actions and reporting on their status, which is much simpler and easier to test.

If you want to know more about the Single Responsibility Principle, the original text by Uncle Bob is the place to start. has a nice example of two responsibilities colliding and interfering with each other. This picture is a great summary of the principle. Finally, there was an interesting debate among Joel Spolsky, Jeff Atwood, and Uncle Bob on the SRP.

Variants of Session-Based Testing

Antony Marcano has been telling us for awhile about the wonders of Session-Based Testing, an idea suggested by Jon and James Bach. It’s a way to record, manage, and audit what clever exploratory testers already do – verify features and find bugs using a rough description of the area to be tested, and strike out in interesting directons if a promising avenue for testing opens up.

We decided finally to try out Session-Based Testing on our HIP project, but with some modifications, and the results so far are very positive – more smiling faces during and after testing, and continued high quality. Here’s a comparison of our process to the standard one described by Antony and the Bachs:

Charter Generation
Antony suggests generating charters before coding (as part of acceptance-test-driven development [PowerPoint]) and refining the charters after coding. It’s also helpful to generate the charters early so you can include the time in iteration planning. He finds that brainstorming sessions tend to last around 30 minutes or so per feature.

The Bachs are pure testers, so generate their charters after development. They don’t say much about how they brainstorm the charters.

We generate charters after completing code for an iteration. We hold one big brainstorming session of about an hour, covering all the features for that iteration (typically 2-4 major features). We include a full day of testing in every iteration plan.

Antony recommends sessions of 25-45 minutes if alone, perhaps up to an hour if pairing. Where possible, it’s helpful to use successful charters as the basis of new automated tests – for instance, if you have a charter for “Entering invalid customer orders”, you might want to create a suite of integration tests that exercise your validation code by entering some of the invalid order types you used during the session.

The Bachs run single-tester sessions of “around 90 minutes”, though it sounds like they may take much longer or much shorter in some cases. They don’t mention test automation. Recorded charters seem to be used mostly for auditing by their customer.

We pair on all our sessions and they last 1-2 hours (we typically do three in one day, similar to the Bachs). We record all our charters on a wiki. We haven’t yet built automated tests corresponding to our successful charters, but hope to do so.