I need to upload quite large files to my app in Azure (Linux, .Net Core 3.1, but also .Net 5.0 fails), but when uploading a file takes more than 2 minutes I get a TCP reset, showing the following error:
Unhandled exception. System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
I looked through many posts with this error, but I do not see this scenario where the limit is 2 minutes and caused by the actual reading of the body stream. I already configured timeout - options.Limits.KeepAliveTimeout = TimeSpan.FromMinutes(20); - but it is not the request duration, it is when reading the stream in HttpProtocol takes longer than 2 minutes. In that case the middle ware is not even reached.
To reproduce I have a console app:
namespace TestUpload
{
public class DurationStream : Stream
{
public DurationStream(TimeSpan duration)
{
// times 512 to prevent being kicked out because of too slow data
Length = 512 * (int) duration.TotalSeconds;
CanRead = true;
CanWrite = false;
CanSeek = false;
}
public override bool CanRead { get; }
public override bool CanSeek { get; }
public override bool CanWrite { get; }
public override long Length { get; }
public override long Position { get; set; }
public override void Flush() { }
public override int Read(byte[] buffer, int offset, int count)
{
var toCopy = Math.Min((int) (Length - Position), count);
for (var i = 0; i < toCopy; ++i)
{
if (i % 512 == 0)
{
Thread.Sleep(TimeSpan.FromSeconds(1));
}
Position++;
}
return toCopy;
}
public override long Seek(long offset, SeekOrigin origin) { throw new NotImplementedException(); }
public override void SetLength(long value) { throw new NotImplementedException(); }
public override void Write(byte[] buffer, int offset, int count) { throw new NotImplementedException(); }
}
internal static class Program
{
private static async Task Main()
{
var httpClient = new HttpClient();
var request = new HttpRequestMessage(HttpMethod.Post, new Uri("http://[myazuresite.com]/upload"))
{
Content = new StreamContent(new DurationStream(TimeSpan.FromSeconds(130)))
};
var start = DateTime.Now;
Console.WriteLine($"Started at {start}");
try
{
await httpClient.SendAsync(request);
}
finally
{
Console.WriteLine($"Finished in {DateTime.Now - start}");
}
}
}
And then as controller:
[HttpPost]
public async Task Upload()
{
await Request.Body.CopyToAsync(Stream.Null);
}
Locally there is no problem running it against Kestrel, and also if I limit the upload to 2 minutes and after that sleep for a minute in the controller it works fine. The request fails after however long the upload is - so if the stream takes 5 minutes it fails after 5 minutes - and never reaches the middle ware. It fails with both http and https.
My hunch is that there is a network component in front of it in Azure that needs some acknowledgement within 2 minutes, but as it reads the whole stream before getting in the middle ware I do not see how to change the behavior. I did not see an easy way to inject a component in the internals of the Kestrel web server.
Suggestions anyone?
