How to enable helpful NullPointerExceptions in gradle test

Question

This problem has been fixed in Gradle 6.6

Original post

I want to put the flag -XX:+ShowCodeDetailsInExceptionMessages to enable helpful NPEs (https://openjdk.java.net/jeps/358) on the tests

I tried

tasks.withType<Test> {
    jvmArgs("-XX:+ShowCodeDetailsInExceptionMessages")
    testLogging {
        setExceptionFormat("full") // Prints the message of the exception
    }
}

But the NPEs have still no messages.

Here's my java version

java -version
openjdk version "14.0.1" 2020-04-14
OpenJDK Runtime Environment (build 14.0.1+7)
OpenJDK 64-Bit Server VM (build 14.0.1+7, mixed mode, sharing)

EDIT WITH MORE RESEARCH

To find out more precisely the origin of the problem, I did some more work. My conclusion is that it doesn't come from my java nor the test framework, it can only be a misconfig of gradle or a bug. Here's what i've done :

I've setup an empty gradle project with this build.gradle.kts

plugins {
    java
}
repositories {
    mavenCentral()
}
dependencies {
    testImplementation("junit:junit:4.13")
}
tasks.withType<Test> {
    jvmArgs("-XX:+ShowCodeDetailsInExceptionMessages") // Helpful NPEs not working :(
    testLogging {
        setExceptionFormat("full") // Prints exception messages
    }
}

And with a test class (src/test/java/TestNPE.java)

import org.junit.*;

public class TestNPE {

    @Test
    public void true_npe() {
        Object o = null;
        o.toString();
    }

    @Test
    public void throw_npe() {
        throw new NullPointerException("My own message");
    }
}

So now

./gradlew test

> Task :test FAILED

TestNPE > throw_npe FAILED
    java.lang.NullPointerException: My own message
        at TestNPE.throw_npe(TestNPE.java:13)

TestNPE > true_npe FAILED
    java.lang.NullPointerException
        at TestNPE.true_npe(TestNPE.java:8)

2 tests completed, 2 failed

This means the framework doesn't do anything special with NPEs.

After that, I retrieved the classpath gradle uses by running test with debug logs. With that, I could run JUnit directly. With -XX:+ShowCodeDetailsInExceptionMessages the helpful NPE message shows up :

java -cp '/Users/apflieger/src/gradle-helpfull-npe/build/classes/java/test:/Users/apflieger/.gradle/caches/modules-2/files-2.1/junit/junit/4.13/e49ccba652b735c93bd6e6f59760d8254cf597dd/junit-4.13.jar:/Users/apflieger/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest-core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-1.3.jar' -XX:+ShowCodeDetailsInExceptionMessages org.junit.runner.JUnitCore TestNPE
JUnit version 4.13
.E.E
Time: 0.006
There were 2 failures:
1) throw_npe(TestNPE)
java.lang.NullPointerException: My own message
    at TestNPE.throw_npe(TestNPE.java:13)
2) true_npe(TestNPE)
java.lang.NullPointerException: Cannot invoke "Object.toString()" because "o" is null
    at TestNPE.true_npe(TestNPE.java:8)

FAILURES!!!
Tests run: 2,  Failures: 2

Answer 1

This is a gradle problem, it will be working from gradle 6.6.

You can try:

@Test
public void true_npe() {
    try {
        Object o = null;
        o.toString();
    } catch (Exception e) {
        e.printStackTrace();
        throw e;
    }
}

The detailed message is in the printed stack trace but it's not in the junit error message

The problem is from gradle, I believe it's using Object Input/Output Stream to pass execution results from test vm to gradle vm. During this exchange the extended message from the original NPE is lost (is a native method).

You can workaround this with a custom junit 5 extension (should be similar for junit 4):

public class HelpfulNullPointerExceptionExtension implements TestExecutionExceptionHandler {
    @Override
    public void handleTestExecutionException(ExtensionContext context, Throwable throwable) throws Throwable {
        if (throwable instanceof NullPointerException) {
            NullPointerException extended = new NullPointerException(throwable.getMessage());
            extended.setStackTrace(throwable.getStackTrace());
            throw extended;
        }
        throw throwable;
    }
}

And register automatically in a META-INF/services/org.junit.jupiter.api.extension.Extension file with content

org.mypackage.HelpfulNullPointerExceptionExtension 

Or, if you care about nested NPE:

public class HelpfulNullPointerExceptionExtension implements TestExecutionExceptionHandler {
@Override
public void handleTestExecutionException(ExtensionContext context, Throwable throwable) throws Throwable {
    Throwable actual = throwable;
    do {
        if (actual instanceof NullPointerException) {
            var field = Throwable.class.getDeclaredField("detailMessage");
            field.setAccessible(true);
            field.set(actual, actual.getMessage());
        }
        actual = actual.getCause();
    } while (actual != null && actual != throwable);
    throw throwable;
}

}