UnfinishedStubbingException when JUnit tests are ran in random order

Question

The following test is one of several tests that fail when I run my tests in random order using this Maven command: mvn -Dsurefire.runOrder=random clean test

@Test
public void ShouldReturnCorrectAccountLoanSumForDebtRatioWhenRedemptionAmountIsNull(){
    AccountVO account = mock(AccountVO.class);
    CustomerGroupInformationVO group = mock(CustomerGroupInformationVO.class);
    when(group.getCustomerIds()).thenReturn(Set.of("199406208123"));
    when(account.getAccountOwners()).thenReturn(List.of((new AccountOwnerVO(null, "199406208123", null))));
    when(account.getAmount()).thenReturn(BigDecimal.valueOf(500000));
    when(account.getRedemptionAmount()).thenReturn(null);

    assertEquals(BigDecimal.valueOf(500000), getAdjustedAccountLoanSumForDebtRatio(account, group, caseClientVO));
}

More specifically this is the line mentioned:

when(account.getAccountOwners()).thenReturn(List.of((new AccountOwnerVO(null, "199406208123", null))));

Any idea what is causing this and how I can fix it? When I run my tests normally using mvn clean install there are no issues at all. The reason I want it to work with a random order is that our build tool seems to use it and it can't build. Like I said it works fine locally.

Answer 1

Because Mockito works through side-effects stored in ThreadLocal variables, it is particularly subject to test pollution: If your tests fail when run in random order, it may be because some previous test left a mock in a state it wasn't expecting to be in. Also, Mockito stubbing relies on observing method calls in a specific order, which can cause odd exceptions if it can't observe the method calls (such as when they're final) or if you interact with a different mock while preparing a thenVerb argument.

One of your first lines of defense is to use validateMockitoUsage, which is meant to be run at the end of every test and confirms that no interaction with Mockito is left unfinished. You could put this in an @After method, but Mockito does so automatically if using MockitoJUnitRunner or MockitoRule. I'd recommend either of those latter options if possible, particularly MockitoRule.

This should help you confirm which test(s) are problematic. Once you're there, try these:

• Double-check that you're not trying to mock final methods without Mockito's opt-in final support. If Mockito can't override your mock method, it won't be able to detect your stubbing calls in its expected order. If any of your methods are written in Kotlin, remember that unlike Java the methods are final unless declared open.
• Your test doesn't seem to use Matchers, but if you do, make sure you use them for all arguments in a method if you use them for any argument in a method.
• Be careful about calling real methods in the middle of stubbing. new AccountOwnerVO(...) shouldn't interact with mocks, but if it does, then Mockito might interpreted it as if your when call never got a thenReturn (when in reality it just hasn't gotten to it yet). Extracting your return value as a local variable is a reasonable step to try.