Capture Screenshots of Selenium Failures

We are constantly fighting a battle with browser-test failures. Our browser tests should be telling us where our application is failing, so we can fix defects quickly and get back to writing more great features – but when you can’t see where an error came from, you can waste hours just trying to reproduce it. The latest weapon we’ve deployed in this battle allows us to capture a screenshot of the browser whenever a test fails.

Unlike other implementations we only want to create a screenshot when a test has failed.

Firstly, our browser tests are written as Java unit tests using the excellent SeleniumRC library. Now capturing a screenshot is really easy as Selenium provides a captureScreenshot() method. The tricky bit is to only capture a screenshot when a test fails. A crude way to achieve this is to wrap every test method with a:

    try { 
      // test here
    } catch (Throwable e) { 
      // capture screenshot here
    }

Instead of doing this we leveraged JUnit’s @RunWith annotation to use a custom runner which catches test failures and trigger the screen capture. To generalise the functionality, our runner will call any method annotated with @AfterFailure.

Here is the source code to our custom runner:

RCRunner.java

import java.util.List;

import org.junit.runners.BlockJUnit4ClassRunner;
import org.junit.runners.model.FrameworkMethod;
import org.junit.runners.model.InitializationError;
import org.junit.runners.model.Statement;

/**
 * JUnit runner which runs all @AfterFailure methods in a test class.
 *
 */
public class RCRunner extends BlockJUnit4ClassRunner {

    /**
     * @param klass Test class
     * @throws InitializationError
     *             if the test class is malformed.
     */
    public RCRunner(Class<?> klass) throws InitializationError {
        super(klass);
    }

    /*
     * Override withAfters() so we can append to the statement which will invoke the test
     * method. We don't override methodBlock() because we wont be able to reference 
     * the target object. 
     */
    @Override
    protected Statement withAfters(FrameworkMethod method, Object target, 
                                   Statement statement) {
        statement = super.withAfters(method, target, statement);
        return withAfterFailures(method, target, statement);
    }

    protected Statement withAfterFailures(FrameworkMethod method, Object target, 
                                          Statement statement) {
        List<FrameworkMethod> failures =
            getTestClass().getAnnotatedMethods(AfterFailure.class);
        return new RunAfterFailures(statement, failures, target);
    }
}

RunAfterFailures.java

import java.util.ArrayList;
import java.util.List;

import org.junit.internal.runners.model.MultipleFailureException;
import org.junit.runners.model.FrameworkMethod;
import org.junit.runners.model.Statement;

public class RunAfterFailures extends Statement {

    private final Statement fNext;

    private final Object fTarget;

    private final List<FrameworkMethod> fAfterFailures;
    
    public RunAfterFailures(Statement next, List<FrameworkMethod> afterFailures,
                            Object target) {
        fNext= next;
        fAfterFailures= afterFailures;
        fTarget= target;
    }

    @Override
    public void evaluate() throws Throwable {
        List<Throwable> fErrors = new ArrayList<Throwable>();
        fErrors.clear();
        try {
            fNext.evaluate();
        } catch (Throwable e) {
            fErrors.add(e);
            for (FrameworkMethod each : fAfterFailures) {
                try {
                    each.invokeExplosively(fTarget, e);
                } catch (Throwable e2) {
                    fErrors.add(e2);
                }
            }
        }
        if (fErrors.isEmpty()) {
            return;
        }
        if (fErrors.size() == 1) {
            throw fErrors.get(0);
        }
        throw new MultipleFailureException(fErrors);
    }

}

AfterFailure.java

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
 * Annotation to mark a method to be called when a test fails.
 *
 */
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface AfterFailure {

}

Here is an example test using the new runner.

MyBrowserTest.java

@RunWith(RCRunner.class)
public class MyBrowserTest {
     
    @Test
    public void myTest() {
        // Test here
    }

    @AfterFailure
    public void captureScreenShotOnFailure(Throwable failure) {
        // Get test method name
        String testMethodName = null;
        for (StackTraceElement stackTrace : failure.getStackTrace()) {
            if (stackTrace.getClassName().equals(this.getClass().getName())) {
                testMethodName = stackTrace.getMethodName();
                break;
            }
        }
        
        selenium.captureScreenshot("/tmp/" + this.getClass().getName() + "."
                                   + testMethodName + ".png");
    }
}

3 thoughts on “Capture Screenshots of Selenium Failures”

  1. Hi,

    Thanks for the detailed explanation. I implemented the above but when I run the test cases as a suite the screen shots are not taken. Could you help.

    Thanks,
    Rahul

Leave a Reply

Your email address will not be published. Required fields are marked *