HTTPClient.PostAsync throws exception only when posting larger file

Question

I have the following code that is used to upload a file to a REST service using POST.

It is working for smaller files but it is throwing an exception for larger ones, by larger I mean around 800 KB.

public async Task<bool> UploadInputFileAsync(FileDTO file)
{
    using (HttpClientHandler httpClientHandler = new HttpClientHandler())
    {
        httpClientHandler.ClientCertificateOptions = ClientCertificateOption.Manual;
        httpClientHandler.ClientCertificates.Add(clientCertificate);
        string url = $"{Configuration.FileService.Url}";

        using (HttpClient httpClient = new HttpClient(httpClientHandler))
        {
            httpClient.DefaultRequestHeaders.TransferEncodingChunked = true;
            MultipartFormDataContent multiForm = new MultipartFormDataContent();

            using (FileStream fs = System.IO.File.OpenRead(Path.Combine(Configuration.FileService.InputFilesPath, file.FileName)))
            {
                multiForm.Add(new StreamContent(fs), "file", file.FileName);

                using (HttpResponseMessage response = await httpClient.PostAsync(url, multiForm))
                {
                    response.EnsureSuccessStatusCode();
                    return true;
                }
            }
        }
    }
}

The exception that is thrown is the following

System.Net.Http.HttpRequestException: Error while copying content to a stream.
 ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
 ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Security.SslStream.<WriteSingleChunk>g__CompleteAsync|210_1[TWriteAdapter](ValueTask writeTask, Byte[] bufferToReturn)
   at System.Net.Security.SslStream.WriteAsyncChunked[TWriteAdapter](TWriteAdapter writeAdapter, ReadOnlyMemory`1 buffer)
   at System.Net.Security.SslStream.WriteAsyncInternal[TWriteAdapter](TWriteAdapter writeAdapter, ReadOnlyMemory`1 buffer)
   at System.Net.Http.HttpConnection.WriteAsync(ReadOnlyMemory`1 source)
   at System.Net.Http.HttpConnection.ChunkedEncodingWriteStream.<WriteAsync>g__WriteChunkAsync|2_0(HttpConnection connection, ReadOnlyMemory`1 buffer)
   at System.IO.Stream.CopyToAsyncInternal(Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
   at System.Net.Http.HttpContent.CopyToAsyncCore(ValueTask copyTask)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpContent.CopyToAsyncCore(ValueTask copyTask)
   at System.Net.Http.MultipartContent.SerializeToStreamAsyncCore(Stream stream, TransportContext context, CancellationToken cancellationToken)
   at System.Net.Http.HttpContent.CopyToAsyncCore(ValueTask copyTask)
   at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)

Using Wireshark I get the following traffic and it appears that the connection is not gracefully closed after FIN,ACK

Wireshark trace

Any help would be appreciated.

LATER EDIT

Thank you for all your help.

It seems that my issue is generated by a bug in SocketsHttpHandler implementation that still exist in .NET Core 3.1 (although they suggest it was fixed in 3.1) that I found described here https://github.com/dotnet/runtime/issues/27415

Solution was to add

AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);

at the beginning of the method.

in the exception itself `An existing connection was forcibly closed by the remote host..` — Hunter Tran, Sep 24 '20 at 04:19
Try increasing the connection time out, so if its taking more time to parse then connection will not close, though in my view issue here could be an exception while parsing the file, which you need to verify, which forcibly close the HttpClient — Mrinal Kamboj, Sep 24 '20 at 04:51

Mikaal Anwar · Answer 1 · 2020-09-24T08:19:30.867

1

The statement "An existing connection was forcibly closed by the remote host" tells us that the problem lies on the receiving end of the request. If we had a malformed request, it would give an error for all files, however, our case is different. Perhaps, the receiving end has enforced a file size limit or a request timeout which isn't allowing the request to complete successfully.

edited Sep 24 '20 at 08:19

answered Sep 24 '20 at 05:35

Mikaal Anwar

1,720
10
21

HTTPClient.PostAsync throws exception only when posting larger file

1 Answers1