29

I have a monstrous Java app (a client of little-known application server GNUEnterprise) and its source, which I can compile back after making some changes to it. The app uses network heavily and I need to monitor each request and response. I could use a sniffer like Wireshark, but the application works with its server over SSL, so not knowing SSL-certificate's private key any sniffed traffic is pretty useless.

What can I do to make every request and response to be logged from the application itself? I need to see all sent and received headers. I don't want to alter all code responsible for network interaction. What I want is to put a code like

Network.setDefaultLogger(myCustomLoggerInstance);

somewhere near start of the app and then in the myCustomLoggerInstance do all the logging I need.

Also, given all network operations are made with URLConnections, I can get response headers with con.getHeaderFields() and request headers with con.getRequestProperties(). But why cookies aren't there? How to dump sent and received cookies in the same way?

EDIT: What I'm trying to reach is to mimic RPC-application's communication with it's server over SSL, say, using curl. For this I need to get detailed log of app's network traffic.

Hnatt
  • 5,767
  • 3
  • 32
  • 43
  • I would modify `OpenJDK`, implement the logger functionality mentioned then sniff the communication at library level. – auselen Sep 03 '13 at 12:22
  • @auselen too hacky. If there's no way to do this on application level (and it seems so), I'll better stick to an external logging proxy server, like suggested by greyfairer. – Hnatt Sep 04 '13 at 08:28
  • Possible duplicate of [How to enable wire logging for a java HttpURLConnection traffic?](https://stackoverflow.com/questions/1445919/how-to-enable-wire-logging-for-a-java-httpurlconnection-traffic) – MCCCS Jul 25 '19 at 15:12

5 Answers5

33

A quick way to log all SSL traffic is to startup java with:

-Djavax.net.debug=all

Or set it as a system property:

System.setProperty("javax.net.debug","all");

Brace yourself. Standard out gets NOISY if you do this!

(*Note: only logs SSL traffic and SSL debug info (handshakes, etc). Regular sockets are not logged.)

Julius Musseau
  • 4,037
  • 23
  • 27
22

Can't you use a logging proxy server, e.g. http://www.charlesproxy.com/ or http://fiddler2.com/? Charles Proxy can also decode AMF requests, Fiddler is the champion for SSL interception.

IIRC you can set system properties http.proxyHost and http.proxyPort to 127.0.0.1 and 8888, and java URLConnections will automatically connect via your logging proxy server, and you get a nice view of your HTTP Traffic.

Or, if the intention is to be able to replay the communication, use JMeter HTTP Proxy, this records in a format that is suitable for replay. It has also built-in support for handling cookies.

If the application uses nice web services, SoapUI also has a monitoring proxy that intercepts web service calls for which you have imported the WSDL.

GeertPt
  • 16,398
  • 2
  • 37
  • 61
  • Oh, and it has SSL traffic sniffing too. I'll definitely try this. – Hnatt Sep 03 '13 at 12:39
  • To allow SSL Sniffing, you need to hack your java truststore, too, see: http://stackoverflow.com/questions/8549749/how-to-capture-https-with-fiddler-in-java – GeertPt Sep 03 '13 at 13:45
  • the app I'm working on already handles that. It has a class that utilizes `HttpsURLConnection.setDefaultHostnameVerifier` and `.setDefaultSSLSocketFactory` to trust all certificates. I'll accept your answer if there will be no application-level solutions suggested, because it actually does what I want, though in a different way. I also found a free alternative to Charles called [webscarab](https://www.owasp.org/index.php/Category:OWASP_WebScarab_Project), which runs on Linux unlike Fiddler. – Hnatt Sep 04 '13 at 08:46
  • I stick to Webscarab proxy. Thanks for pointing me in the right direction! – Hnatt Sep 09 '13 at 08:28
  • 2
    you had me to google 'IIRC', means "If I Recall Correctly" – Jalal Sordo Nov 27 '20 at 00:49
5

I dug a little further, and apparently, HttpURLConnection comes with java.util.logging enabled by default.

See related questions: Java: Display request of an HttpURLConnection before sending and How to enable wire logging for a java HttpURLConnection traffic?

Java.util.logging was Sun's (failed) attempt at standardizing logging providers in Java 1.4. You can bridge it to SLF4J via the jul-to-slf4j module if you want to avoid tweaking system properties.

But I would still prefer Fiddler or JMeter proxy though :-)

Community
  • 1
  • 1
GeertPt
  • 16,398
  • 2
  • 37
  • 61
2

I just want to post a snippet of code that you can use at starting point. I do not have a"global" logger but with small changes to existing code you can reach your goal. I "log" on standard output, since your problem can be split into two pieces: getting the info from UrlConnection and store it for further reference.

This code logs the request:

public class URLConnectionReader {

  public static void main(String[] args) throws Exception {
  URL oracle = new URL("http://www.google.com/");
  URLConnection yc = oracle.openConnection();

  String headerName = null;
  for (int i = 1; (headerName = yc.getHeaderFieldKey(i)) != null; i++) {
    if (headerName.equals("Set-Cookie")) {
      String cookie = yc.getHeaderField(i);
      System.out.println("Cookie  ::  " + cookie);
    } else {
      System.out.println("Header: "+ yc.getHeaderField(i));
    }
  }

  BufferedReader in = new BufferedReader(new InputStreamReader(yc.getInputStream()));
  String inputLine;
  while ((inputLine = in.readLine()) != null)
    System.out.println(inputLine);
  in.close();
}

}

saving all this data to one or more files is not hard. If you believe that I'm on right way for you problem, I would be glad to edit the answer with other details as needed.

giampaolo
  • 6,906
  • 5
  • 45
  • 73
  • Thanks, but it looks like `getHeaderFieldKey(i)` and `getHeaders()` for some reason skip `Set-Cookie` headers. Anyway, I'm looking for a way to set a global network logger for entire application, not having to put code snippets everywhere network interaction takes place. Also, with your solution I have to read the connection twice. – Hnatt Aug 30 '13 at 08:05
  • And of course saving the log to a file is not a problem. So I'd split my question in two following parts: 1) Setting global logger; 2) Logging the traffic without reading or sending responses and requests twice. – Hnatt Aug 30 '13 at 08:59
  • Oops, my bad. If you call `getHeaders()` after sending a request on a `URLConnection` instance, it doesn't open the reading stream second time, and you actually can see `Set-Cookie` headers there. So +1 for you, sorry for all the wrong claims. What's left is a way to set proxy for all network connections (I mean OOP pattern Proxy, not a proxy server). – Hnatt Aug 30 '13 at 09:09
  • Why don't you simple extend URLConnection and override getConnection? Put in your constructor your logger instance. Just a guess. – giampaolo Aug 30 '13 at 09:49
  • @tapo: I'm looking for something like that, but if I extend `URLConnection`, I'll need to substitute every `URLConnection` constructor with my `CustomURLConnection`. It wouldn't be a big problem if I knew for sure that `URLConnection` is always instantiated with a constructor. But there may be methods returning `URLConnection` instance, like `URL.openConnection()`. So I need to extend and replace `URL` too and maybe something else. I don't want to mess a lot with program's code, especially inserting snippets all over it. – Hnatt Aug 30 '13 at 12:05
1

If you want to log network traffic & you have URLConnection objects then the problem is already solved!
If you want to log at stream level, you need to write a little wrapper on top of your I/O streams & log all data before transfer them to lower network layers, otherwise you can use connection objects directly to get required info & log them.
For managing cookies, you should use java.net.HttpCookie which is introduced in java 6. Only thing you have to do is to create an instance of CookieManager & set it as default cookie manager:

CookieManager cookieManager = new CookieManager();
CookieHandler.setDefault(cookieManager);

& you can use it for managing connection cookies!

Ehsan Khodarahmi
  • 4,772
  • 10
  • 60
  • 87
  • Originally I've searched for something like this global cookie manager object, only for all network actions, but then I decided that using local logging proxy server is enough. I don't even need to meddle with source code in this case. By the way, one can use custom `CookieManager` to log incoming and outcoming request headers, not only cookies. `CookieManager`'s methods `get` and `put` have `requestHeaders` and `responseHeaders` correspondingly as their second argument of type `List`. I wonder only if these methods are called at every request & response or only when cookies are used. – Hnatt Sep 09 '13 at 08:37