How to read full response from HttpURLConnection?

Question

I make some proxy server in andorid which modify http headers, it works ok, but I have to forward full response to 'top layer'.
How I can read whole response (all headers, content, everything) from HttpURLConnection?

HttpURLConnection httpURLConnection;
URL url = new URL(ADDRESS);
httpURLConnection = (HttpURLConnection) url.openConnection();
// add headers, write output stream, flush
if (httpURLConnection.getResponseCode() == HttpsURLConnection.HTTP_OK)
{
    Map<String, List<String>> map = httpURLConnection.getHeaderFields();
    System.out.println("Printing Response Header...\n");

    for (Map.Entry<String, List<String>> entry : map.entrySet())
    {
        System.out.println("Key : " + entry.getKey() + " ,Value : " + entry.getValue());
    }

    return new DataInputStream(httpURLConnection.getInputStream());
}

In getInputStream I received only content it is possible to have some stream with whole reposne?

No, there's no way to do this with `HttpUrlConnection`. But you can simulate it by getting the response code, response message and all the headers. — Sotirios Delimanolis, Jan 07 '14 at 15:53

score 35 · Accepted Answer · answered Jan 07 '14 at 16:08

There's no way to dump the full HTTP response directly using the HttpURLConnection, but you can use its various method to reconstruct it. For example,

HttpURLConnection httpURLConnection;
URL url = new URL("http://www.google.com");
httpURLConnection = (HttpURLConnection) url.openConnection();
StringBuilder builder = new StringBuilder();
builder.append(httpURLConnection.getResponseCode())
       .append(" ")
       .append(httpURLConnection.getResponseMessage())
       .append("\n");

Map<String, List<String>> map = httpURLConnection.getHeaderFields();
for (Map.Entry<String, List<String>> entry : map.entrySet())
{
    if (entry.getKey() == null) 
        continue;
    builder.append( entry.getKey())
           .append(": ");

    List<String> headerValues = entry.getValue();
    Iterator<String> it = headerValues.iterator();
    if (it.hasNext()) {
        builder.append(it.next());

        while (it.hasNext()) {
            builder.append(", ")
                   .append(it.next());
        }
    }

    builder.append("\n");
}

System.out.println(builder);

prints

200 OK
X-Frame-Options: SAMEORIGIN
Transfer-Encoding: chunked
Date: Tue, 07 Jan 2014 16:06:45 GMT
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
X-XSS-Protection: 1; mode=block
Expires: -1
Alternate-Protocol: 80:quic
Set-Cookie: NID=67=OIu8_xhcxE-UPCSfIoTINvRyOe4ALVhIqan2NUI6LMdRkSJHTPGvNkYeYE--WqPSEPK4c4ubvmjWGUyFgXsa453KHavX9gUeKdzfInU2Q25yWP3YtMhsIhJpUQbYL4gq; expires=Wed, 09-Jul-2014 16:06:45 GMT; path=/; domain=.google.ca; HttpOnly, PREF=ID=4496ed99b812997d:FF=0:TM=1389110805:LM=1389110805:S=jxodjb3UjGJSZGaF; expires=Thu, 07-Jan-2016 16:06:45 GMT; path=/; domain=.google.ca
Content-Type: text/html; charset=ISO-8859-1
Server: gws
Cache-Control: private, max-age=0

You can then get the InputStream and print its content too.

I am trying to get the inputStream but when I am printing the stream to a text, it is not complete text. I guess it is half downloaded only. — Darpan, Sep 21 '14 at 00:04
your code doesn't establish a connection. httpURLConnection.connect(); missing. — Error, Sep 15 '16 at 05:39
@Error If it didn't establish a connection, how did it get a response (code, headers, body)? No, this code obviously does establish a connection, it just does it implicitly inside the `getResponseCode()` call which internally calls `getInputStream()`. — Sotirios Delimanolis, Sep 15 '16 at 14:45
I barely read the source! i just look to doc unfortunately doc did highlight that https://docs.oracle.com/javase/7/docs/api/java/net/HttpURLConnection.html#getResponseCode() , it does that is true http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/net/HttpURLConnection.java#HttpURLConnection.getResponseCode%28%29 for future readers — Error, Sep 15 '16 at 14:56
To turn this into a "full" HTTP header, change the first line to `builder.append("HTTP/1.1 ").append(httpURLConnection.getResponseCode())` code, and put builder.append(`\n`) after the for loop, before the println. The `HTTP/1.1` might not match what was originally sent, but it will probably "work" if it needs to be given to anything expecting a http response. — Pod, Mar 12 '19 at 14:14

score 6 · Answer 2 · edited May 23 '17 at 12:25

6

It wasn't obvious to me at first when I looked for a similar problem, so I but I found a solution.

Read the body response:

readFullyAsString(connection.getInputStream(), "UTF-8");

and this comes from : https://stackoverflow.com/a/10505933/1281350

public String readFullyAsString(InputStream inputStream, String encoding) throws IOException {
        return readFully(inputStream).toString(encoding);
    }

    private ByteArrayOutputStream readFully(InputStream inputStream) throws IOException {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        byte[] buffer = new byte[1024];
        int length = 0;
        while ((length = inputStream.read(buffer)) != -1) {
            baos.write(buffer, 0, length);
        }
        return baos;
    }

edited May 23 '17 at 12:25

Community

1
1

answered Oct 25 '16 at 13:41

Greg

689
2
8
23

2

This doesn't fully answer the question. The question is how to get the full response including HTTP headers. This just answers how to read an InputStream which is the easy part. – Necreaux Nov 14 '17 at 15:52
2

See here for a one-liner solution, valid for Java 8: https://stackoverflow.com/a/45659529/3585768 – Jonathan Benn Apr 11 '18 at 13:52

How to read full response from HttpURLConnection?

2 Answers2

Linked