Why my JUnit error collector is not reporting the error?

Question

I am trying to report error using JUnit error collector. Although my assertion is failing, error is not reported in JUnit. But I am getting the "error" message in console.

@Rule
public ErrorCollector errcol = new ErrorCollector();

@Then("^Business alert message on the screen$")
public void Business_alert_message_on_the_screen(Result_Update) throws Throwable {
    if (userType.equals("Admin")) {
        try {
            Assert.assertEquals("Update button is not present for the admin user", true, Result_Update);

        } catch (Throwable t) {
            errcol.addError(t);
            System.out.println("Error");
        }
     }
}

Answer 1

According to JUnit:

The ErrorCollector rule allows execution of a test to continue after the first problem is found

 errcol.addError(t);//only adds the error to the ErrorCollector

This means that the test continues after collecting the error.

You should add:

 errcol.checkThat(...); //will pass/fail the test

See examples:

https://junit.org/junit4/javadoc/4.12/org/junit/rules/ErrorCollector.html (Updated)

https://gist.github.com/cb372/2419626

Answer 2

tl;dr : Make sure your test class doesn't extend TestCase.

I had a similar problem when I was using JUnit 4 with IntelliJ IDEA. I naïvely selected a base class of TestCase in the dialog, which was the default for JUnit 3, because I figured "it'd be nice to have those handy this#assert* methods" (the default for JUnit 4 is null). The bad code (which didn't work) is below:

public class SassCompilerTest extends TestCase {
    @Rule
    public ErrorCollector collector = new ErrorCollector();

    @Test
    public void testCompiler() throws IOException {
        collector.checkThat(true, CoreMatchers.equalTo(false));
    }
}

However, in JUnit 4, that prevented a lot of features from working. Removing the parent class fixed the test:

public class SassCompilerTest {
    @Rule
    public ErrorCollector collector = new ErrorCollector();

    @Test
    public void testCompiler() throws IOException {
        collector.checkThat(true, CoreMatchers.equalTo(false));
    }
}

The solution was suggested to me by a comment in the issue with Cucumber mentioned by @StefanBirkner in another answer. After reading that, I tried extending ErrorCollector to make the ErrorCollector#verify public and call it from an @After method, but the @After method wasn't getting called, which made me realize something was either wrong with the TestRunner (which was IntelliJ's default) or the Test itself.

Answer 3

The Cucumber runner does not support @Rule because it extends ParentRunner and not BlockJUnit4ClassRunner (see source code of the runner). There is already an issue for supporting ErrorCollector.