4

I'm using the Apache HttpClient 4.2.3 in my application. We store the response of an HTTP call like so:

HttpResponse httpResponse = (DefaultHttpClient)httpClient.execute(httpRequest);

The response body is an InputStream in the 4.x API:

InputStream responseStream = httpResponse.getEntity().getContent();

My problem is I need to read the response body as a string and as a byte[] at various points in the application. But the InputStream used by Apache is an EofSensorInputStream, which means once I reach the stream EOF, it gets closed. Is there anyway I can get the string and byte[] representations multiple times and not close the stream?

I've already tried wrapping the byte array in a new ByteArrayInputStream and setting that as the request body, but it doesn't work since my response body can reach a few gigs. I've also tried this, but I noticed the original response stream still gets closed.

Any pointers would be welcome.

EDIT: On a related note, it would be also be great if I could find the length of the InputStream either without consuming the stream or by reversing the consumption.

Community
  • 1
  • 1
Karthik V
  • 1,033
  • 1
  • 14
  • 29

1 Answers1

3

1 . I think you have somewhat conflicting requirements:

a)

it doesn't work since my response body can reach a few gigs

b)

Is there anyway I can get the string and byte[] representations multiple times and not close the stream

If you do not have enough memory this is not possible.

Btw, another way to get the response as bytes is EntityUtils.byte[] toByteArray(final HttpEntity entity).

Do you really need N-gigs String? What are you going to do with it?

2 .

it would be also be great if I could find the length of the InputStream

httpResponse.getEntity().getContentLength()

3 . Since the response does not fit into the memory I would suggest to save it into a file (or temp file). Then set up InputStream on that file, and then read it as many times as you need.

Vitaly
  • 2,760
  • 2
  • 19
  • 26
  • 1
    IOUtils or EntityUtils would give you the byte[], wouldn't since they read through the entire stream, the stream gets closed in this case (EofSensorInputStream). That means I can't call the getAsByteArray(InputStream) again on the response stream. Also, about the memory issue, I probably can afford to take a large byte[] once in memory, but not more than that. – Karthik V Apr 26 '13 at 05:03
  • @Karthik.V Once you get byte[] from EntityUtils.toByteArray() you will have your single copy of the data as you wanted. You can also wrap it into ByteArrayInputStream and it should not cause OutOfMemoryError since ByteArrayInputStream does not copy byte[]. Btw, you should not keep response for so long time, remember this is a response from a remote server. How are you going to re-read it? That would cause the server re-send you N-gigs again? See my item 3 in the answer about how to re-read large response several times. – Vitaly Apr 26 '13 at 06:15