6

Is there an extension for JUnit4 which allows for marking some tests as "expected to fail"?

I would like to mark the test for current features under development with some tag, for instance @wip. For these tests I would like to ensure that they are failing.

My acceptance criteria:

Scenario: A successful test tagged @wip is recorded as failure
    Given a successful test marked @wip
    When the test is executed
    Then the test is recorded as failure.

Scenario: A failing test tagged @wip is recorded as fine
    Given a failing test tagged @wip
    When the test is executed
    Then the test is recorded as fine.

Scenario: A successful test not tagged @wip is recorded as fine
    Given a successful test not tagged @wip
    When the test is executed
    Then the test is recorded as successful.

Scenario: A failing test not tagged with @wip is recorded as failure
    Given a failing test not tagged with @wip
    When the test is executed
    Then the test is recorded as failure.
Alex Schwartz
  • 85
  • 1
  • 4
  • 1
    possible duplicate of [Mark unit test as an expected failure in JUnit](http://stackoverflow.com/questions/4055022/mark-unit-test-as-an-expected-failure-in-junit) –  Jan 31 '14 at 15:14

2 Answers2

8

Short answer, no extension will do that as far as I know and in my opinion it would defeat the whole purpose of JUnit if it would exist.

Longer answer, red/green is kind of sacred and circumventing it shouldn't become a habit. What if you accidentally forgot to remove the circumvention and assume that all tests passed?

You could make it expect an AssertionError or Exception.

@wip
@Test(expected=AssertionError.class)
public void wipTest() {
   fail("work in progress");
}

Making a shortcut in your IDE for that shouldn't be too hard. Of course I was assuming you tag the test with an annotation in the source code.

In my opinion what you are asking is against JUnit's purpose, but I do understand the use for it.

An alternative would be to implement a WIPRunner with the WIP annotation and somehow make it accept failures of tests with the WIP annotation.

If you are integrating with a BDD framework I would suggest a way to let it run the unit tests you marked @wip seperately and decide within your BDD methods if the result is ok.

Gressie
  • 507
  • 2
  • 7
  • 2
    My justification for wanting to do this: Assume QA writes an integration test for a feature that is not implemented yet. Then I'd like the test to xFail (expected-fail) if the feature is still not implemented and Fail if it suddenly is (because it means I should revisit the test and turn it into normal unit test). – user7610 Jul 14 '16 at 13:58
  • Sometimes it is easier to mark down instances of a bug, or degraded behaviour, and then invert the result as a sort of "pin-down" test that can be later reversed when the bug is fixed, so you get regression tests for free. It's unfortunate that `expected` got removed in JUnit5, as I've been trying to find a suitable alternative that doesn't involve hacking extensions. https://blog.ganssle.io/articles/2021/11/pytest-xfail.html – Jason Nguyen Jul 15 '23 at 16:42
4

The @Ignore annotation says not to bother with the result.

Wes
  • 6,697
  • 6
  • 34
  • 59
  • 3
    oh yes, but I would like to bother about the result. In particular, a test tagged as @wip should be executed, and it must fail. – Alex Schwartz May 04 '11 at 21:37
  • Okay define fail? Do you mean you expect an exception? – Wes May 04 '11 at 21:39
  • What you describe in the description is exactly the reverse of a normal test. Why not reverse all assertions in your test, then you have the desired effect. – zenog Jan 10 '13 at 18:19
  • @zenog I'm not sure if that comment was directed at me? I never really got the question I think. – Wes Jan 22 '13 at 15:47