Narrative-style testing with Bildungsroman

Bildungsroman, n. A novel whose principal subject is the moral, psychological, and intellectual development of a usually youthful main character.

Narrative is a framework we developed at TIM Group in order to make it easier to write acceptance-level tests in Java. An example Narrative test looks like this:

@Test public void
adds_two_numbers() {
    Given.the( operator).was_able_to( press('2'))
                        .was_able_to( press('+'))
                        .was_able_to( press('2'));

    When.the( operator).attempts_to( press('='));

    Then.the( operator).expects_that( the_displayed_value(), is("4"));
}

One essential idea here is that tests are broken up into “Given”, “When” and “Then” phases. Another is that when we are writing behaviour-driven tests we are often concerned with the actions of some particular agent – a user, whose credentials and preferences affect how the system will respond to their requests. Finally, this style of testing involves writing re-usable definitions of the various operations a user can perform. We trade off the slight cumbersomeness of extracting test steps into definitions against the usefulness of having these definitions available for re-use at a later stage.

Bildungsroman is an adaptation of the Narrative framework for use in Scala projects. In Bildungsroman, the above example might look like this:

describe("A calculator") {
  it("adds two numbers") {
    calculatorContext.verify(for {
      _ <- givenThe(operator) { presses('2') }
                   .andThen   { presses('+') }
                   .andThen   { presses('2') }

      _ <- whenThe(operator)  { presses('=') }

      _ <- thenThe(operator)  { seesTheDisplayedValue('4') }
    } yield ())
  }
}

There is some additional noise here, imposed by the use of a for-comprehension to thread the steps together, but the fundamental concepts remain the same.

The step definitions for the above code might look like this:

val calculatorRef = Ref[Calculator]("the calculator")

def presses(key: Char) = { operator: User =>
  calculatorRef.map(_.press(operator, key))
}

def seesTheDisplayedValue(display: String) = { operator: User =>
  calculatorRef.map(_.readDisplay(operator))
}

A calculatorRef is a key into the context that all steps share. A Ref is a type-safe way to interact with this context, either by reading a value from it or by writing a value to it. Here’s an example, taking two numbers from the context and writing their sum back into the context:

for {
  n1 <- numberRef1
  n2 <- numberRef2
  sum  <- numberRef3 := (n1 + n2)
} yield sum

Note the use of the := operator to assign a value to the slot indicated by the Ref. Refs have map and flatMap defined on them, which enables them to be assembled together into for-comprehensions like the above. They are in fact the building-blocks of a state monad.

The state monad is an abstraction which enables us to thread a context through a series of operations that each have the opportunity to “modify” that context. An important point here is that the context’s data is in fact immutable: operations “modify” it by returning a new, updated context, rather than by mutating the context that is passed to them. This gives us the ability to do “imperative programming in the small”, or to model mutable variables in an immutable environment.

In the case of the calculator example above, we need a calculator object to hand in order to carry out the steps of pressing buttons and reading the display. This object is supplied through the context, which we pre-populate:

val calculatorContext = Context(calculatorRef -> new Calculator())

calculatorContext.validate(canAddTwoNumbers)

An interesting point here is that we can define a single Narrative-style test, such as canAddTwoNumbers, and execute it against multiple contexts – a useful technique for testing invariants, e.g. the toy calculator and the scientific calculator both add numbers in the same way.

Contexts are also designed for re-use and extension. Suppose we have a standard context for writing tests involving users and groups:

val userGroupContext = Context(
  userRepositoryRef -> new UserRepository(),
  groupRepositoryRef -> new GroupRepository()
)

If we want to extend that context to include a Roles repository, we can use the original userGroupContext as a basis:

val userGroupAndRoleContext = userGroupContext.extend(
  roleRepositoryRef -> new RoleRepository()
)

Equally importantly, narratives themselves compose. The type of a narrative test which finally yields a value of type A is GWTState[A]. If we have two tests, one which yields an A and one which yields a B, we can compose them in a for-comprehension to get a test yielding (A, B), which has the type GWTState[(A, B)] and can in turn be composed with other tests:

for {
  valueOfTypeA <- testYieldingA
  valueOfTypeB <- testYieldingB
} yield (valueOfTypeA, valueOfTypeB)

The second test will inherit the context returned by the first, and return its own modified context. This offers a powerful way of composing larger test steps out of smaller ones.

Bildungsroman is free and open source, hosted at the youDevise/TIM Group GitHub account, and ready for hacking. We’ll be experimenting with it over the coming weeks to see if it offers a good Scala substitute for the benefits we used to get by using Narrative in Java, – readability of tests, re-usability of actions and a structured framework for behaviour-driven test development.