Gojko Adzic held a super introduction to the Cucumber testing tool the other night at SkillsMatter. Below are some notes from the session. We’re considering taking up Cucumber in a couple of ways, including using it as part of our evaluation of new folks who want to join us as testing experts. (We want to know how good a candidate is likely to be at writing automated tests, and even if you have never done test automation, you should be able to write a Cucumber test as it’s just plain English. Or Welsh.)
When it first appeared, it seemed the main advantage of Cucumber was that you could use tables in addition to RSpec given/when/then, which didn’t seem that cool at first. But as Gojko’s worked on it, he’s found that it’s great for staying out of your way as you build your tests:
- Generates code for you
- Supports lists and tabular data
- Tagging for types of tests
- JUnit XML output, so easy to integrate with CI tools and friendly to non-Ruby languages
The basic structure of a Cucumber test is plain text, like this:
Feature: Hello World
In order to ensure that my installation works
As a developer
I want to run a quick Cucumber test
Scenario: Hello World
Given The Action is Hello
When The Subject is World
Then The Greeting is Hello, World.
Cucumber forces you to use the good Given/When/Then structure that keeps a clear distinction among components:
- Your scenario is readable by humans (see above) and is scripted by…
- …the step definitions, which are in Ruby (or your favourite alternative language) and can be reused to test many…
- …domain classes, which do the actual work.
Your step definitions can be refactored along with the domain classes to help insulate the tests against code changes. Another way to say this is that you get to separate your high-level activities (the scenario) from your implementation of those activities (the step definitions).
You can write your scenario first with no step definitions and run Cucumber with no errors! Instead it will tell you that your tests are undefined and suggest how to start creating the appropriate step definitions. This makes real behaviour-driven development possible – write the scenario, then the step definitions, then finally get to your domain objects.
Cucumber will give you various output formats so you can see the red/green loveliness in PDF, XML, HTML, and more. It lets you use your favourite language including Java, Ruby, C#, and even LOLCODE!
You can avoid wordy repetition of the same scenario again and again in some clever ways, including scenario outlines (which let you use a compact set of rows to show a group of scenarios using a common format), lists of objects (which specify multiple values per Given/When/Then steps), and common setup (using a “Background” keyword).
- Console only. But there are plenty of ways to run it in a GUI.
- No IDE support (yet). Apparently there is a VisualStudio plugin. Can Eclipse be far behind?
- No way to add images or other documentation to your test, as you can in Fit and Fitnesse.
You can get long-term maintenance problems if you don’t keep the semantics straight – your scenarios tell you *what* is being tested, not *how* it is done. Antony Marcano points out that you have to watch out for step definitions that drift from their titles – don’t let your “send message” step get defined to log or write to the database.