Does Apache Commons HttpClient support GZIP?

Question

Does the library Apache Commons HttpClient support Gzip? We wanted to use enable gzip compression on our Apache server to speed up the client/server communications (we have a php page that allows our Android application to sync files with the Server).

Answer 1

Apache HttpClient 4.1 supports content compression out of the box along with many other features that were previously considered out of scope.

Answer 2

If your server is able to provide GZIPped content, with Apache Http client 4.1 all you need is to use

org.apache.http.impl.client.ContentEncodingHttpClient

which is a subclass of DefaultHttpClient.

This client will also add headers saying that it accepts GZIPped content.

Answer 3

Since 4.1, Apache HttpClients handles request and response compression.

• You don't need to compress request, no need to set any "Accept-Encoding" in request headers.
• It automatically handles response decompression as well, no need to handle Decompression of response.
• Till 4.3 it handles gzip and deflate. You can check ResponseContentEncoding api doc here.

Just use:

HttpClients.custom()

which uses:

HttpClientBuilder.create()

If you want to check in library goto HttpClientBuilder it uses RequestAcceptEncoding & ResponseContentEncoding

You can disable it through "disableContentCompression()"

HttpClient httpClient = HttpClients.custom()
                .setConnectionManager(cm)
                .disableContentCompression() //this disables compression
                .build();

Please make sure if you add any interceptor it can override that, use it carefully.

HttpClient httpClient = HttpClients.custom()
                .setConnectionManager(cm)
                .setHttpProcessor(httpprocessor) //this interceptor can override your compression.
                .build();

Answer 4

It has no support for this out-of-the-box, and it seems unlikely to be added to HttpClient 3.x (see rather bitchy JIRA issue here). You can, however, do it by adding custom request readers and manual request/response stream handling, layered on top of the basic library, but it's fiddly.

It seems you can do it with HttpClient 4, but not without some effort.

Pretty shoddy, if you ask me, this stuff really should be easier than it is.

Answer 5

Here is the sample scala code which uses java apache-http-client library

 def createCloseableHttpClient(): CloseableHttpClient = {
    val builder: HttpClientBuilder = HttpClientBuilder.create
    val closableClient = builder.build()
    closableClient
  }

  def postData(data: String): Unit = {
    val entity = EntityBuilder.create()
      .setText(data)
      .setContentType(ContentType.TEXT_PLAIN)
      .gzipCompress()
      .build()
    val post = new HttpPost(postURL + endPoint)
    post.setEntity(entity)
    post.setHeader("Content-Type", "application/gzip")
    val client = createCloseableHttpClient()
    client.execute(post)
    client.close()
  }

Answer 6

For those (like me) who cannot for other reasons update to Apache 4.x.x and need to have their Apache (btw below was first implemented for version 4.3.6 where it was apparently NOT supported "out of the box" though this may be of course since we always use custom ones having other specific demands on our httpclients ["out of the box" could of course be fine if you are fully up to date and ok with what this concept will actually imply - for instance the default retry policy, the SSL-hostname validation, default timeouts and so on would not have worked for us - and if you use custom instead, you have to understand what you are doing which is always better anyway in the long run..?]). Anyway, the solution will work for versions pre 4.x.x too - I'm adding something more in the lines of an example following Garry's fine response above. Building with maven, and accommodating compressed responses in general I've added

  <dependency>
      <groupId>org.brotli</groupId>
      <artifactId>dec</artifactId>
      <version>0.1.2</version>
  </dependency>

to my pom.xml. Then in order to handle all these three common compression methods (gzip, deflate and brotli) I've added interceptors when setting up my custom Apache HttpClient - like this:

private static final String GZIP = "gzip";
private static final String DEFLATE = "deflate";
private static final String BR = "br";
private static final String ACCEPT_ENCODING = "Accept-Encoding";
private static final String COMMA_SPACE = ", ";
private static final String SUPPORTED_COMPRESSION_VARIANTS = new StringBuilder(GZIP).append(COMMA_SPACE).append(DEFLATE).append(COMMA_SPACE).append(BR).toString();

..
                   HttpClients.custom()..
                   .addInterceptorFirst((HttpRequestInterceptor) (request, context) -> { // add accepting compressed headers _always_
                    if (!request.containsHeader(ACCEPT_ENCODING)) {
                        request.addHeader(ACCEPT_ENCODING, SUPPORTED_COMPRESSION_VARIANTS);
                    }
                    })
                    .addInterceptorFirst((HttpResponseInterceptor) (response, context) -> {
                    HttpEntity entity = response.getEntity();
                    Header ceheader = entity != null ? entity.getContentEncoding() : null;
                    if (ceheader != null) {
                        HeaderElement[] codecs = ceheader.getElements();
                        for (int i = 0; i < codecs.length; i++) {
                            if (codecs[i].getName().equalsIgnoreCase(GZIP)) { // handling gzip
                                response.setEntity(new GzipDecompressingEntity(response.getEntity()));
                            } else if (codecs[i].getName().equalsIgnoreCase(DEFLATE)) { // handling deflate
                                response.setEntity(new DeflateDecompressingEntity(response.getEntity()));
                            } else if (codecs[i].getName().equalsIgnoreCase(BR)) { // handling brotli
                                try (BufferedReader br =
                                             new BufferedReader(new InputStreamReader(new BrotliInputStream(response.getEntity().getContent())))) {
                                     response.setEntity(new StringEntity(br.lines().collect(Collectors.joining())));
                                }
                            }
                        }
                    }
                  }).build();

So - the accept header for these will always be added on outgoing requests and compressed responses are handled by the response interceptor. Works fine.

Answer 7

It doesn't support it out of the box but you can transform entity of returned HttpResponse into uncompressed one by calling

val entity = new GzipDecompressingEntity(response.getEntity)

then proceed with entity.getContent as always.

Answer 8

Custom Protocol Interceptors may help as well.

Disclaimer: I haven't tried this yet.