Why would a shared HttpClient timeout more often?

Question

I am writing some code to test some Wowza streams by making HTTP calls to various URLs related to the stream and checking for a 200 status. I wrote a simple method to call httpClient.GetAsync and return a test result:

  public async Task<TestResult> DoHttpGetTestAsync(string name, string url)
    {
        if (string.IsNullOrEmpty(url)) return new TestResult { TestName = name, Status = "Skipped", Description = "No url" };
        try
        {                
            var response = await client.GetAsync(url);
            if (response.IsSuccessStatusCode) return new TestResult { TestName = name, Status = "Success", Description = response.StatusCode.ToString() + ' ' + url };
            return new TestResult { TestName = name, Status = "Failed", Description = response.ReasonPhrase + ' ' + url };
        }
        catch (Exception ex)
        {
            return new TestResult { TestName = name, Status = "Failed", Description = ex.Message + ' ' + url };
        }
    }

I then use this to test a handful of URLs:

   if (camera.SupportsHighRes) results.Add(await DoHttpGetTestAsync("Get high res Wowza Status", cameraUrls.HighResStatusUrl, "normal"));
   if (camera.SupportsLowRes) results.Add(await DoHttpGetTestAsync("Get low res Wowza Status", cameraUrls.LowResStatusUrl, "normal"));
   if (camera.SupportsLowRes) results.Add(await DoHttpGetTestAsync("Get low res thumbnail", cameraUrls.LowResThumbnailUrl));
   if (camera.SupportsHighRes) results.Add(await DoHttpGetTestAsync("Get high res thumbnail", cameraUrls.HighResThumbnailUrl));

These work as expected. However if I add a call to the HLS feed:

 if (camera.SupportsLowRes) results.Add(await DoHttpGetTestAsync("Get low res HLS feed", cameraUrls.LowResHLSFeedUrl));
 if (camera.SupportsHighRes) results.Add(await DoHttpGetTestAsync("Get high res HLS feed", cameraUrls.HighResHLSFeedUrl));

Then the previous calls to get status and thumbnails timeout more, throwing a TaskCanceledException. It is as though getting the HLS feed interferes with the other calls and causes them to never complete. Note I am not processing the HLS at all - just checking for a 200 to come back. Looking at it in Fiddler I see this response from the HLS call:

HTTP/1.1 200 OK
Access-Control-Expose-Headers: Date, Server, Content-Type, Content-Length
Access-Control-Allow-Headers: Content-Type, User-Agent, If-Modified-Since, Cache-Control, Range
Date: Thu, 26 Oct 2017 18:54:25 GMT
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: OPTIONS, GET, POST, HEAD
Content-Type: application/vnd.apple.mpegurl
Access-Control-Allow-Credentials: true
Accept-Ranges: bytes
Server: WowzaStreamingEngine/4.5.0
Cache-Control: no-cache
Content-Length: 125

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=276673,CODECS="avc1.77.31",RESOLUTION=427x240
chunklist_w684280577.m3u8

I don't see any redirect or anything that I would suspect of causing HttpClient to get bogged down.

I would suspect some bug with Wowza or throttling of some sort, except that if I use a unique instance of HttpClient, I do not see the problem. I've tested by making these calls in a loop both with a shared HttpClient and with a HttpClient-per-test.

The code above has timeouts more often with:

static HttpClient client = new HttpClient();

and it has few or no timeouts with:

private HttpClient client = new HttpClient();

The latter is discouraged, however, since it can lead to socket exhaustion under heavy use.

What about this would cause HttpClient to timeout more often?

You need to show how you use `HttpClient`. A single static instance should be fine if you use it correctly. — Crowcoder, Oct 26 '17 at 19:41
I have a feeling your requests are timing out. What is your timeout property set to in your HttpClient? https://msdn.microsoft.com/en-us/library/system.net.http.httpclient.timeout(v=vs.110).aspx — Rogala, Oct 26 '17 at 19:49
I've tried several things to get it stable, At some point the client cancels after two minutes (regardless the timeout setting). Eventually I gave up and switched to WebClient, which happens to be a lot faster. Here's an interesting article 'Bugs and Documentation Errors in .NET's HttpClient Frustrate Developers': https://www.infoq.com/news/2016/09/HttpClient — , Oct 26 '17 at 19:56
@Crowcoder I've updated to include this, but it really doesn't do much. I'd prefer the single static instance, but only see this timeout behavior when I do. — Daniel, Oct 26 '17 at 19:58
`HttpClient` can be used under heavy load just fine, as long as you do it correctly. — xxbbcc, Oct 26 '17 at 19:59
@Rogala - yes, they are timing out. I've tested with various values, but they timeout regardless of whether it's set for 10s or the default. So it is a timeout, but something about httpclient seems to be part of the cause (ie not just random timeouts from the server) — Daniel, Oct 26 '17 at 19:59
That looks fine since you are not changing default headers and such. Maybe it is the service you are connecting to that cannot handle the load. I'm not familiar with Wowza. — Crowcoder, Oct 26 '17 at 20:00
@Daniel What's the quota? Maybe you have reach that and they are not responding nicely to you? — Rogala, Oct 26 '17 at 20:03
@Crowcoder that's what I would lean to as well, except that I _dont_ see the issue with a HttpClient-per-instance. In both cases, the load on the service is the same - 6 GETs. Only with the static HttpClient do I see the timeout though. — Daniel, Oct 26 '17 at 20:04
@Rogala - I don't think it's a Wowza throttling thing - In both cases, the load on the service is the same - 6 GETs. Only with the static HttpClient do I see the timeout. Also, I would expect a 400 error, not a timeout if it were a server issue. — Daniel, Oct 26 '17 at 20:07
@Daniel Does this help? https://learn.microsoft.com/en-us/dotnet/framework/network-programming/managing-connections I got that from https://stackoverflow.com/questions/31735569/what-is-httpclients-default-maximum-connections — Rogala, Oct 26 '17 at 20:08
Well, you got me. Even if you completely botched best practices you shouldn't notice with only six requests. — Crowcoder, Oct 26 '17 at 20:18
Rogala - I had high hopes for this. The default is 4 connections per server, and this behavior occurs when I add a 5th connection to the same server. However, I modified the ConnectionLimit both ways shown in the doc and still get the same behavior. — Daniel, Oct 26 '17 at 20:27

Why would a shared HttpClient timeout more often?

0 Answers0