JUnit displays 0.4 sec, Infinitest a Slow Test Warning for a 4 ms test

Question

I have this test:

@Test
public void findsPackageClasses() throws ClassNotFoundException, IOException {
    final long startTime = System.currentTimeMillis();
    Class<?>[] classes = finder.getPackageClasses(ClassFinderStubsPackageMarker.class.getPackage());
    Arrays.sort(classes, (c1, c2) -> c1.getName().compareTo(c2.getName()));
    assertArrayEquals(STUB_CLASSES, classes);
    System.out.println(String.format("Test duration: %d ms", System.currentTimeMillis() - startTime));
}

When I run it as a JUnit test in Eclipse, the JUnit report regularly displays 0.4-0.5 sec for the duration of this test. Infinitest also issues a Slow Test Warning for it. However, the duration calculation using currentTimeMillis() always results 4-5 ms, printed to the console. What's going on?

I'd like to get rid of the warning by eliminating the slow part(s) - moving to integration test - but the speed reports seem contradictory. In addition, the only slow part could be finder.getPackageClasses() but its file system dependency is mocked out with Mockito (set up to return class literals). I've even put temporarily code to throw exception when live code is hit (which I've also tested by temporarily removing the mock, this way the exception was effectively thrown in the live code without mocking), gaining confirmation that the mock works when it's injected. The other slow part might be getPackage() which I haven't investigated yet thoroughly (calculating delta with currentTimeMillis() returns 0 ms for it). Anyway, the speed reports are permanently contradictory, which is very strange. Have you ever encountered a similar phenomenon?

score 0 · Answer 1 · answered Feb 06 '20 at 16:35

0

I just encountered this same problem. The actual @Test runs in 40ms but Infinitest and Gradle both report a runtime close to 900ms. Upon inspection of my test I realized that the @BeforeEach cost was huge ~90% even tho that was only setting up a bunch of mockito mocks.

From what I can tell there is no way to get rid of this warning aside from changing the Inifitest preferences. It doesn't make sense to me that setup/teardown costs are factored into the runtime of the test. At the very least it would be great to have some annotations that allow you to configure this on a per test basis.

answered Feb 06 '20 at 16:35

Mark Cesnik

19
1

Sorry that I wasn't around 3 years ago to answer sooner. – Mark Cesnik Feb 06 '20 at 17:05
Thank you for your contribution. AFAIK, based on the four phase test approach to structuring tests and considering the inline vs implicit setup patterns as well as the whole purpose of having Fast tests F.I.R.S.T., setup/teardown costs, so `@BeforeEach` as well, definitely are to be factored into the run time of the test. – Steve Feb 08 '20 at 01:59
In addition, mocking itself is exactly one of the ways that should make the tests fast. – Steve Feb 08 '20 at 02:09
In my case specifically the ```@BeforeEach``` was instantiating a new mock() using mockito. When I changed the code such that the instantiation happened upon declaration and ```@BeforeEach``` simply reset() the mocks the test ran nearly 90% faster. – Mark Cesnik Feb 10 '20 at 20:33

JUnit displays 0.4 sec, Infinitest a Slow Test Warning for a 4 ms test

1 Answers1