-1

I've rolled my own code to retrieve data from remote URLs using a HttpURLConnection. First I had to write custom code to handle gzipped content encoding, now I am encountering web servers that are serving up RSS feeds with a chunked Transfer-Encoding. I'm finding myself reimplementing a bunch of stuff that I'm sure has already been done.

I'm sure there is some library that will automatically handle the various ways that data can be returned from a web server and just give me an InputStream, can anyone recommend one (must be LGPL or similar license)?

Ideally I'm looking for a library that is currently maintained and which has a modern well-designed API.

In some cases I won't know the Content-type that will be returned in advance, and so may need to handle what is returned differently depending on this.

sanity
  • 35,347
  • 40
  • 135
  • 226
  • 1
    My first idea would be to look at [Apache HttpClient](http://hc.apache.org/) – I don't know for sure it supports all your requirements, but it might considering how long it's been around for. – millimoose Jun 26 '12 at 20:14

1 Answers1

0

I know Apache Commons HttpComponents supports Content-Encoding as of 4.1+ (see here), using ContentEncodingHttpClient. DecompressingHttpClient should also work for 4.2+ (they have slightly different behavior in conjunction with caching).

Like millimoose said, it may not have everything. But I think it will be a better starting place.

Community
  • 1
  • 1
Matthew Flaschen
  • 278,309
  • 50
  • 514
  • 539
  • But what if you don't know whether the content will be encoded or compressed before you attempt to retrieve it? – sanity Jun 26 '12 at 20:31
  • @sanity, I believe that's fine. `DecompressingHttpClient` will ask for compressed content using Accept-Encoding, but will check the Content-Encoding to determine if the response actually was compressed. – Matthew Flaschen Jun 26 '12 at 20:34