JUnit integration tests should report HTTP request/response details

Question

I'm writing some Integration Tests (ITs) that are being run using Maven Failsafe plugin.

In short, these ITs perform an HTTP call and analyze the JSON response to ensure certain elements are present.

When a test fails, I would like to be able to see the details of the HTTP request and response (headers, body, etc.), not just the assertion failure message.

If my test looks something like this:

public class FooBarTest {

    MyHttpClient httpClient;

    @Before
    public void setupHttpClient(){
       this.httpClient = ...
    }

    @Test
    public void testFooBarBaz(){
       Response response = this.httpClient.get("http://some/url");
       Assert.assertEquals(200, response.getStatus());
       String json = response.getBody();
       Assert.assertEquals("baz", evalJsonPath(json, "foo.bar.id"));
    }
}

When the test is run from command line (through Maven), I want to be able to see the assertion error and additionally the request and response details. I assume this requires printing to System.out/err but it's better done through some Logging system.

Additionally, I want the same information to be available in the test TXT report file that surefire/failsafe produces:

-------------------------------------------------------------------------------
Test set: com.sample.FooBarTest
-------------------------------------------------------------------------------

REQUEST: {...}
RESPONSE: {...}

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 3.273 sec <<< FAILURE! - in com.sample.FooBarTest
testFooBarBaz(com.sample.FooBarTest)  Time elapsed: 3.27 sec  <<< ERROR!
junit.framework.AssertionFailedError: expected:<200> but was:<404>
     at junit.framework.Assert.fail(Assert.java:47)
     at junit.framework.Assert.failNotEquals(Assert.java:283)
     at junit.framework.Assert.assertEquals(Assert.java:64)
     at junit.framework.Assert.assertEquals(Assert.java:195)
     at junit.framework.Assert.assertEquals(Assert.java:201)
     at com.sample.FooBarTest.testFooBarBaz(FooBarTest.java:XX)

Finally, the same details should be present in some fashion in the XML report, in a way that Jenkins displays this information as well when drilling down into failed tests pages.

If at all possible, I only care about printing this information when there are failed tests.

How can I accomplish this? I've started looking into these options, but more guidance would be appreciated

• Custom JUnit reporter, runner or listener
• JUnit @Rules (Method/Class Rule, ErrorCollector, etc.)
• Using some special logger
• ...

PS. I'm not looking for a solution that simply writes such details to a separate file since the I consider the implications of that to be somewhat less user-friendly.

Thanks!

Answer 1

You can use Hamcrest (Why should I use Hamcrest-Matcher and assertThat() instead of traditional assertXXX()-Methods) with a custom Matcher<Response> (how to implement a hamcrest matcher, Is there a simple way to match a field using Hamcrest?).

With a little work on the Matchers, this will allow you to write your test as:

assertThat(response, hasStatus(200));
assertThat(response, hasJsonPathEval("foo.bar.id", eq("baz"));

and any failure will include the toString representation of your response.

This is functionally equivalent to:

assertEquals(response.toString(), 200, response.getStatus());

but makes clearer that you expect the response and a description of the test to be shown on failure.

Answer 2

All you need to do is to make a custom error message to print whatever you want

 assertEquals( getResponseDetails(response), 200, response.getStatus());
 ...
 assertEquals( getResponseDetails(response), "baz", evalJsonPath(json, "foo.bar.id"));

Where getResponseDetails() is a method that you write.

private String getResponseDetails(Response response){
   StringBuider builder = new StringBuilder();

   buider.append("header = '").append(response.getHeader()).append("'\n");
   ...//similar code for body etc

   return builder.toString();

}

This String will be written to wherever you have configured the tests output to go in your Maven configuration and will only be printed when the test fails.

Or optionally since it appears Response may be a custom class that you created, you can make Response#toString() contain the information you want in which case you just have to do this:

 assertEquals( response, 200, response.getStatus());
 ...
 assertEquals( response, "baz", evalJsonPath(json, "foo.bar.id"));

As a side note: I notice from your stack trace you are using junit.framework.Assert which is the old JUnit 3 library. You should use JUnit 4's org.junit.Assert instead