Using PowerMock to mock System.class in multiple tests not working as expected

Question

Class under test:

public class SystemCaller {
  public String callSystem() {
    return System.getenv("test-this");
  }
}

Test 1:

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.powermock.api.mockito.PowerMockito;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.PowerMockRunner;

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.is;

@RunWith(PowerMockRunner.class)
@PrepareForTest({SystemCaller.class, System.class})
public class SystemTestOne {

    @Before
    public void start() throws Exception {
        PowerMockito.mockStatic(System.class);
        PowerMockito.when(System.getenv("test-this")).thenReturn("hello-one");
    }

    @After
    public void stop() {}

    @Test
    public void verifySystemTestOne() throws Exception {
        assertThat(new SystemCaller().callSystem(), is("hello-one"));
    }
}

Test 2:

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.powermock.api.mockito.PowerMockito;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.PowerMockRunner;

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.is;

@RunWith(PowerMockRunner.class)
@PrepareForTest({SystemCaller.class, System.class})
public class SystemTestTwo {

    @Before
    public void start() throws Exception {
        PowerMockito.mockStatic(System.class);
        PowerMockito.when(System.getenv("test-this")).thenReturn("hello-two");
    }

    @After
    public void stop() {}

    @Test
    public void verifySystemTestTwo() throws Exception {
        assertThat(new SystemCaller().callSystem(), is("hello-two"));
    }
}

If run individually, each test passes on its own. However, when I run all tests, the tests fail. And the error varies. Sometimes, the assertion expects hello-two, but gets hello-one. At other times, callSystem() just returns null!

Is there a setup issue here? Or, is this because I'm mocking a static method (on System class in this case) that's causing the issue?

My dependencies:

      "org.apache.commons" % "commons-lang3" % "3.5" ::
      "com.squareup.okhttp3" % "okhttp" % "3.6.0" ::
      "com.google.code.gson" % "gson" % "2.8.0" ::
      "com.novocode" % "junit-interface" % "0.8" % "test->default" ::
      "junit" % "junit" % "4.12" % "test" ::
      "org.hamcrest" % "hamcrest-core" % "1.3" % "test" ::
      "org.hamcrest" % "hamcrest-library" % "1.3.RC2" % "test" ::
      "com.squareup.okhttp3" % "mockwebserver" % "3.8.0" % "test" ::
      "org.powermock" % "powermock-core" % "1.6.6" % "test" ::
      "org.powermock" % "powermock-api-mockito" % "1.6.6" % "test" ::
      "org.powermock" % "powermock-module-junit4" % "1.6.6" % "test" ::

UPDATE 1:

OK, the suggestion from @kevin-welker, to use EnvironmentVariables actually worked. However, I'm seeing a weird issue where, when I run the JUnit tests from Intellij IDEA (using JUnit Run Configuration), everything works, but when I run them via sbt (thru ./sbt test) they fail with the same error :(.

I'm leveraging https://github.com/sbt/junit-interface to run these tests, but can't seem to get them to work.

UPDATE 2:

Alright! Thanks to @kevin-welker and @stefan-birkner, I got the tests to work as expected! Since there's only a handful of tests I need to run, and they're really fast, I'm OK losing out on the parallelism!

Does `sbt` run your tests in parallel? Then this problem is possible because the map that stores the environment variables is a singleton. — Stefan Birkner, Oct 19 '17 at 04:51
@StefanBirkner . . https://stackoverflow.com/questions/11899723/how-to-turn-off-parallel-execution-of-tests-for-multi-project-builds solved it — kodeninja, Oct 19 '17 at 22:38

score 2 · Accepted Answer · answered Oct 18 '17 at 16:48

2

There's a micro library to help with mocking system classes called System Rules. Use is as simple as

@Rule
public final ProvideSystemProperty myPropertyHasMyValue
    = new ProvideSystemProperty("MyProperty", "MyValue");

answered Oct 18 '17 at 16:48

Kevin Welker

7,719
1
40
56

Same error when using `EnvironmentVariables`. One test passed, the other failed: [error] Test SystemTestOne.verifySystemTestOne failed: [error] Expected: is "hello-one" [error] but: was "hello-two" [error] Failed: Total 2, Failed 1, Errors 0, Passed 1, Skipped 0 [error] Failed tests: [error] SystemTestOne [error] (test:test) sbt.TestsFailedException: Tests unsuccessful – kodeninja Oct 18 '17 at 17:07
are you doing the `environmentVariables.set()` in the @Before as in your OP examples, or are you doing that in the test method as shown in the System Rules examples? – Kevin Welker Oct 18 '17 at 17:58
In the `@Before` method. Should I move it to the `test()` method? – kodeninja Oct 18 '17 at 18:03
Moved `env.set(...)` to the `test()` method, but got the same error. – kodeninja Oct 18 '17 at 18:10
Kevin, please see my update. Your suggestion actually worked, but I'm not able to get it to pass when running the tests via `sbt test`. – kodeninja Oct 18 '17 at 22:43

score 0 · Answer 2 · answered Oct 18 '17 at 16:32

0

You cannot mock System and other system classes. That's usually why you create wrappers like your SystemCaller class that will only contains the untestable logic and that you can mock where they are used in the code.

More info here: https://github.com/powermock/powermock/wiki/Mock-System

answered Oct 18 '17 at 16:32

Nyamiou The Galeanthrope

1,178
9
23

I was following https://stackoverflow.com/questions/46290503/having-trouble-mocking-system-getenv-calls-in-junit to mock `System.class`. Are you saying that solution is incorrect? – kodeninja Oct 18 '17 at 17:15
Yes this solution is incorrect as you cannot prepare for test System. – Nyamiou The Galeanthrope Oct 19 '17 at 09:10

score 0 · Answer 3 · answered Oct 18 '17 at 16:40

0

You can try the PowerMockito.doCallRealMethod().when(System.class); after your test (@After or @AfterClass), it should reset the mock of System.class.

answered Oct 18 '17 at 16:40

staszko032

802
6
16

Tried that, no dice. – kodeninja Oct 18 '17 at 16:56

Using PowerMock to mock System.class in multiple tests not working as expected

3 Answers3