HttpClient: Conditionally set AcceptEncoding compression at runtime

Question

We are trying to implement user-determined (on a settings screen) optional gzip compression in our client which uses HttpClient, so we can log and compare performance across a number of different calls over a period of time. Our first attempt was to simply conditionally add the header as follows:

HttpRequestMessage request = new HttpRequestMessage(Method, Uri);
if (AcceptGzipEncoding)
{
     _client.DefaultRequestHeaders.AcceptEncoding.Add(new System.Net.Http.Headers.StringWithQualityHeaderValue("gzip"));
}

//Send to the server
result = await _client.SendAsync(request);

//Read the content of the result response from the server
content = await result.Content.ReadAsStringAsync();

This created the correct request, but the gzipped response was not decompressed on return, resulting in a garbled response. I found that we had to include the HttpClientHandler when constructing the HttpClient:

HttpClient _client = new HttpClient(new HttpClientHandler
    { 
        AutomaticDecompression = DecompressionMethods.GZip
    });

This all works well, but we'd like to change whether the client sends the Accept-Encoding: gzip header at runtime, and there doesn't appear to be any way to access or change the HttpClientHandler after it's passed to the HttpClient constructor. In addition, altering the headers of the HttpRequestMessage object doesn't have any effect on the headers of the request if they are defined by the HttpClientHandler.

Is there any way to do this without recreating the HttpClient each time this changes?

Edit: I've also tried to modify a reference to the HttpClientHandler to change AutomaticDecompression at runtime, but that's throwing this exception:

This instance has already started one or more requests. Properties can only be modified before sending the first request.

Just curious, now that you've seen how handy automatic decompression is, why *not* recreate the client each time that setting changes? That's exactly what I'd do here, unless there's a very good reason not to. — Todd Menier, Feb 27 '15 at 14:59
@ToddMenier That's a very valid question. It's something we're considering, but it will require a bit of restructuring I think, because the same HttpClient is tied into a number of areas. It's a moderately sized application inherited from earlier developers so we just have to manage changes carefully. We'll be looking at it this week. Thanks for the comments and help. — pcdev, Mar 02 '15 at 03:09

score 14 · Answer 1 · answered Feb 27 '15 at 14:45

You're almost there with the first example, you just need to deflate the stream yourself. MS's GZipSteam will help with this:

HttpRequestMessage request = new HttpRequestMessage(Method, Uri);
if (AcceptGzipEncoding)
{
     _client.DefaultRequestHeaders.AcceptEncoding.Add(new System.Net.Http.Headers.StringWithQualityHeaderValue("gzip"));
}

//Send to the server
result = await _client.SendAsync(request);

//Read the content of the result response from the server
using (Stream stream = await result.Content.ReadAsStreamAsync())
using (Stream decompressed = new GZipStream(stream, CompressionMode.Decompress))
using (StreamReader reader = new StreamReader(decompressed))
{
    content = reader.ReadToEnd();
}

Hmm, it turns out that this answer works unless the server decides not to (or isn't configured to) gzip the response, then it falls over. Unfortunately there's no easy way to get at the "Content-Encoding" header from the HttpResponseMessage object [see this post](https://social.msdn.microsoft.com/Forums/windowsapps/en-US/cb7417b5-ca3e-44f6-a272-9e2f8fc5d9b8/portable-httpclient-hides-contentlength-and-contentencoding-headers-with-gzip-encoding?forum=wpdevelop), so it looks like recreating the HttpClient is really the only way to do it. Thanks again for your input. — pcdev, Mar 06 '15 at 01:08

score 8 · Answer 2 · edited May 23 '17 at 11:47

If you want to use the same HttpClient and only want to enable compression for some requests, you are not able to use automatic decompression. When automatic decompression is enabled, the framework also resets the Content-Encoding header of the response. This means that you are unable to find out whether the response was really compressed or not. By the way, also the Content-Length header of the response matches the size of the decompressed content if you turn on automatic decompression.

So you need to decompress the content manually. The following sample shows an implementation for gzip-compressed content (as also shown in @ToddMenier's response):

private async Task<string> ReadContentAsString(HttpResponseMessage response)
{
    // Check whether response is compressed
    if (response.Content.Headers.ContentEncoding.Any(x => x == "gzip")) 
    {
        // Decompress manually
        using (var s = await response.Content.ReadAsStreamAsync())
        {
            using (var decompressed = new GZipStream(s, CompressionMode.Decompress))
            {
                using (var rdr As New IO.StreamReader(decompressed))
                {
                    return await rdr.ReadToEndAsync();
                }
            }
        }
    else
        // Use standard implementation if not compressed
        return await response.Content.ReadAsStringAsync();
}

Our code is using NSCachedUrlResponse to save on some bandwidth, and depending on which server we connected to, some scripts were not running. After a quick SO search I found the above method, that was all I needed for my code. — Code Knox, Apr 23 '18 at 17:35

score 2 · Accepted Answer · answered Mar 06 '15 at 01:12

As per the comments above, recreating the HttpClient is really the only (robust) way to do this. Manual decompression can be achieved but it seems to be very difficult to reliably/efficiently determine whether the content has been encoded or not, to determine whether to apply decoding.

HttpClient: Conditionally set AcceptEncoding compression at runtime

3 Answers3