11

We have a .NET WorkerService that is running in the background pending trigger from the Event Hubs to push data to an API webservice but we are running into this issue:

Any ideas what could be causing this ?

​​​​ExceptionSource: "System.Net.Http", ExceptionType: "System.Threading.Tasks.TaskCanceledException: The operation was canceled. 

System.IO.IOException: Unable to read data from the transport connection: Operation canceled.

System.Net.Sockets.SocketException (125): Operation canceled 

End of inner exception stack trace --- 
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) 
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token) 
at System.Net.Security.SslStream.<FillBufferAsync>g__InternalFillBufferAsync|215_0[TReadAdapter](TReadAdapter adap, ValueTask`1 task, Int32 min, Int32 initial) 
at System.Net.Security.SslStream.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory`1 buffer) 
at System.Net.Http.HttpConnection.FillAsync() 
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed) 
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) 

--- End of inner exception stack trace --- 
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) 
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) 
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) 
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) 
at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) 
at Client.MyClient.UpdateAsync(DataRequestUpdate DataRequest, CancellationToken cancellationToken) in /src/Client/Client.cs:line 1232 
at Client.Repository.Update(Request data) in /src/Client/Repository.cs:line 32 
at WorkerService.EventProcessor.Update(UpdateRequest request) in /src/WorkerService/EventProcessor.cs:line 390", Message: "The operation was canceled." }​
Yan Sklyarenko
  • 31,557
  • 24
  • 104
  • 139
chad
  • 627
  • 1
  • 8
  • 24
  • 3
    Hi, do you use timeout for your `HttpClient`? Also, please take a look [github issue](https://github.com/dotnet/runtime/issues/33280) – DarkSideMoon Feb 16 '21 at 08:16
  • @DarkSideMoon This is really helpful. I will look into this and see if it's the same issue. Thanks! – chad Feb 16 '21 at 09:01
  • No problem :) Also, if you can - could you please vote another my answers Thanks a lot! – DarkSideMoon Feb 17 '21 at 07:52
  • Did anyone reach to a conclusion? The Github issue seems was resolved without clear explanation on why this happens, though clearly reproducible in Linux environments. – marcofo88 Apr 21 '21 at 09:56
  • @marcofo88 I have managed to solve it by increasing the HttpClient timeout. The default was 100 secs and I've found that it took more than 100 secs for some reasons. The problem disappears after increasing it to 10 minutes. HttpClient httpClient = new HttpClient(); httpClient.Timeout = TimeSpan.FromMinutes(10); – chad Apr 22 '21 at 13:01
  • Interesting, though that might have other unintended implications. I wonder why the difference in the environment. Thanks a lot for getting back! – marcofo88 Apr 23 '21 at 14:00
  • @chad, for me it was also a timeout. Please write it down as an answer, if it worked for you it is a valid answer. – Maurice Klimek Jul 13 '21 at 13:56
  • 1
    @MauriceKlimek Thanks for confirming. I have wrote down the answer. – chad Jul 21 '21 at 12:43

1 Answers1

18

The default .NET httpclient timeout is 100 secs. If your client takes more than 100 secs, it will result in the above error. To solve this, simply increase the timeout value.

HttpClient httpClient = new HttpClient(); 
httpClient.Timeout = TimeSpan.FromMinutes(10); //Eg. 10mins timeout
chad
  • 627
  • 1
  • 8
  • 24