How to debug everything that goes over the wire (http)

Question

I'm trying to debug some Rest service in Glassfish 4 and I want to see exactly what is being sent over the wire (HTTP connection). Is there a way to make Glassfish print that out?

My project uses Apache HTTP Client and I know is it possible to configure it there, but I couldn't get it to work with our logging framework(java.util.logging).

EDIT: Alternatively is there a way to debug Jersey that will make it printout this info?

Answer 1

You don't specify if you have control over the client/front end or not, or if this is just over the backend.

A few immediate thoughts come to mind:

1. Use a packet sniffer/analyzer like WireShark (as already suggested). Works great, the only problem is that will probably generate significantly more data than you really want to see/analyze.

2. Use a proxy like Charles or Fiddler. These can be configured to show/log exactly what data you want/need. The disadvantage is that they are proxies (ie: have to be in the middle of your connection) and this can cause configuration difficulties if you are putting it in front of your server. Usually much easier to configure on the client end, and just push your client traffic through it. Fiddler is free, but only Windows based (alpha version available for Mac and Linux). Charles has a trial version, or otherwise is 50$ (if memory serves). I also do not know if Fiddler and/or Charles will run in headless mode.

3. Set up your own Http Proxy server and log everything through there. Not truly an ideal solution, but an option none the less.

4. Add a filter to your application to log all HTTP data coming in/out, but that gets tricky/messy as well as incoming Http request bodies are single read, so they need to be cached for multiple reads/accesses (see Http Servlet request lose params from POST body after read it once and https://stackoverflow.com/a/17129256/827480 for a good proposed solution to this issue)

Answer 2

What http client version you are using? For http client 4 it is in log4j:

log4j.category.org.apache.http.wire=DEBUG

Or with util logging:

org.apache.http.wire.level=FINEST

Answer 3

By using the Filters we can track the response, Add this filter configuration in web.xml file

<filter-mapping>
    <filter-name>Test</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter>
    <filter-name>Test</filter-name>
    <filter-class>net.vsspl.Test</filter-class>
</filter>

The filter calss is

import java.io.IOException;
import java.util.ArrayList;

import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletResponse;

public class Test implements Filter {

  public void init(FilterConfig filterConfig) throws ServletException {}

  public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
    chain.doFilter(request, response);
    HttpServletResponse httpServletResponse = (HttpServletResponse) response;
    System.out.println("Status Code" + httpServletResponse.getStatus());
    System.out.println("Name of the character encoding" + httpServletResponse.getCharacterEncoding());
    System.out.println("Content type" + httpServletResponse.getContentType());
    ArrayList<String> headers = (ArrayList<String>) httpServletResponse.getHeaderNames();
    for (String header : headers) {
      System.out.println(header + " Header value" + httpServletResponse.getHeader(header));
    }
  }

  public void destroy() {}

}

Answer 4

Others have suggested wireshark, which is an excellent tool, but somewhat cumbersome to use. You typically have to catch the traffic, transfer it to your desktop, and then take a look.

My most commonly used tool is ngrep, which combines tcpdump style packet filters with grep style filters on the packet content, and an output display style which is adequate for debugging most HTTP issues.

$ sudo ngrep -d tun0 '^(GET|POST|HEAD|HTTP)' port 80
interface: tun0 (10.104.1.6/255.255.255.255)
filter: (ip or ip6) and ( port 80 )
match: ^(GET|POST|HEAD|HTTP)
####
T 10.104.1.6:39700 -> 93.184.216.119:80 [AP]
  GET / HTTP/1.1..User-Agent: Wget/1.14 (linux-gnu)..Accept: */*..Host: examp
  le.com..Connection: Keep-Alive....                                         
##
T 93.184.216.119:80 -> 10.104.1.6:39700 [A]
  HTTP/1.1 200 OK..Accept-Ranges: bytes..Cache-Control: max-age=604800..Conte
  nt-Type: text/html..Date: Fri, 24 Oct 2014 11:21:44 GMT..Etag: "359670651".
  .Expires: Fri, 31 Oct 2014 11:21:44 GMT..Last-Modified: Fri, 09 Aug 2013 23
  :54:35 GMT..Server: ECS (cpm/F9D5)..X-Cache: HIT..x-ec-custom-error: 1..Con
  tent-Length: 1270....<!doctype html>.<html>.<head>.    <title>Example Domai
  n</title>..    <meta charset="utf-8" />.    <meta http-equiv="Content-type"
   content="text/html; charset=utf-8" />.    <meta name="viewport" content="w
  idth=device-width, initial-scale=1" />.    <style type="text/css">.    body
   {.        background-color: #f0f0f2;.        margin: 0;.        padding: 0
  ;.        font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, san
  s-serif;.        .    }.    div {.        width: 600px;.        margin: 5em
   auto;.        padding: 50px;.        background-color: #fff;.        borde
  r-radius: 1em;.    }.    a:link, a:visited {.        color: #38488f;.      
    text-decoration: none;.    }.    @media (max-width: 700px) {.        body
   {.            background-color: #fff;.        }.        div {.            
  width: auto;.            margin: 0 auto;.            border-radius: 0;.    
          padding: 1em;.        }.    }.    </style>    .</head>..<body>.<div
  >.    <h1>Example Domain</h1>.    <p>This domain is established to be used 
  for il

I also recommend a more obscure tool called chaosreader available from http://www.brendangregg.com/Chaosreader/chaosreader0.94. It'll turn a tcpdump file into a browsable set of html and other files, showing you all the traffic in a very accessible form. Much easier than wireshark when you are more interested in the content of web communications than the breakdown into packets.

Answer 5

If you have access to the code, you can either:

1. Update your logging to print out info from the HTTP Client.

2. Add an HTTP Interceptor, something like the following:

   public class HttpLogInterceptor extends HandlerInterceptorAdapter {
   
   @Autowired
   private RequestDumper requestDumper;
   
   @Override
   public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
       requestDumper.doLogRequest(request, response);
       return true;
   };
   
   @Override
   public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView)
           throws Exception {
       requestDumper.doLogResponse(request, response);
   }
   

   }

and use the following, or similar to this:

Answer 6

If you want to spit out the Request and Response from the REST Client code perspective, please use the client side logging using com.sun.jersey.api.client.filter.LoggingFilter might help.