Developing custom FindBugs detectors – a test-driven approach.

There are precious few examples ([1] and [2], below, being good ones) on the interwebs showing how to develop custom detectors for FindBugs, a handy feature of the static analysis tool I’ve blogged about previously. There are even fewer (zero?) examples showing how to create automated tests for your new detector. The implication being that the way to test a detector is to run the FindBugs program and manually inspect the results. As you can probably guess this quickly becomes tedious, and the feedback loop for your code changes is frustratingly long. Fortunately,  there is now a better way. Using a new open-source utility from youDevise, test-driven-detectors4findbugs, this blog post will show how to test-drive development of your custom FindBugs detector. Thus shortening the feedback loop, guiding development, and leaving behind a helpful set of automated unit tests to catch later regressions.

For the sake of this discussion, lets say we want to raise a bug against classes whose names are too long – not that FindBugs is the best tool for this job, but it’s a trivial enough ‘bug’ to demonstrate test-driving a FindBugs detector.

It’s often best to write some examples (or ‘benchmarks’) to test your detector with, for example, a class which you would wish FindBugs to report a bug for, and one which would you wouldn’t. If you had a more complicated code pattern you wished to detect, you would add a benchmark for each interesting case. For now, the ‘class name too long’ bug only requires a couple of cases. These could be:

class ExampleClassWithAnAllowedName { }
class ExampleClassWithANameThatIsTooLongForThisSillyDetector { }

Next, we can write a failing unit test. This will use functionality from the test-driven-detectors4findbugs, which is a JAR file which needs to be added to your testing classpath. Although there are a couple of tests that you could write at this point, I’ll show just one. This assumes a JUnit test, but the utility should work with other unit test frameworks which use the AssertionError mechanism for failing tests, e.g. TestNG:

import com.youdevise.fbplugins.tdd4fb.DetectorAssert;

@Test public void
raisesBugAgainstClassWithLongName() throws Exception {
    // Obtain a BugReporter instance from this method
    BugReporter bugReporter = DetectorAssert.bugReporterForTesting();

    // And pass the same BugReporter to your detector
    Detector detector = new CustomClassNameLengthDetector(bugReporter);

    // Next assert that your detector has raised a bug against your benchmark class

This will fail to compile, since theres currently no Detector class which conforms to the specification required by FindBugs. So we can, write out the template for your detector class. For FindBugs to execute your custom detector, it must implement the Detector interface, as well as having a convention-based constructor signature,  since the detector is instantiated with reflection. Thus your template should look roughly like this:

import edu.umd.cs.findbugs.BugInstance;
import edu.umd.cs.findbugs.BugReporter;
import edu.umd.cs.findbugs.Detector;

public class CustomClassNameLengthDetector implements Detector {
    private final BugReporter bugReporter;

    public CustomClassNameLengthDetector(BugReporter bugReporter) {
        this.bugReporter = bugReporter;

    @Override public void report() { }
    @Override public void visitClassContext(ClassContext classContext) { }

The test should now compile, but fail, since the CustomClassNameLengthDetector does not raise a bug. Now you’re ready to add in the real functionality, while listening to the tests telling you when your code runs as you expect, without you having to manually execute the whole FindBugs process through the Eclipse, Swing or command-line interfaces.

Filling in the implementation of the detector is not particularly interesting for this example, so I’ll wrap it up there. To see the whole code listing, you can check out the source on GitHub. There are also some examples of  more sophisticated assertions, using Hamcrest matchers.

Additional info on test-driven-detectors4findbugs:

  • a pre-built JAR is available for download from the download section of the GitHub homepage
  • released under the OSI-approved MIT license
  • has runtime dependencies on FindBugs 1.3.9 and Hamcrest 1.1+
  • although requiring version 1.3.9 of FindBugs to test against, detectors will work with FindBugs versions since (at least) 1.3.7

Writing your own FindBugs detectors is a great way to catch bugs you and your team haven’t written yet. If you’re developing one, hopefully test-driven-detectors4findbugs will make the process quite a bit less painful.