Why does StreamReader.ReadToEnd work but not Stream.Read?

Question

I am trying to get the body of a request in an ASP.NET Core controller as a byte[] array. Here is what I initially wrote:

var declaredLength = (int)request.ContentLength;
byte[] fileBuffer = new byte[declaredLength];

request.Body.Read(fileBuffer, 0, declaredLength);

This code works, but only for small requests (around ~20KB). For larger requests it fills up the first 20,000 or so bytes in the array, then the rest of the array is empty.

I used some code in the top answer here, and was able to read the entire request body successfully after rewriting my code:

var declaredLength = (int)request.ContentLength;
byte[] fileBuffer = new byte[declaredLength];

// need to enable, otherwise Seek() fails
request.EnableRewind();

// using StreamReader apparently resolves the issue
using (var reader = new StreamReader(request.Body, Encoding.UTF8, true, 1024, true))
{
    reader.ReadToEnd();
}

request.Body.Seek(0, SeekOrigin.Begin);
request.Body.Read(fileBuffer, 0, declaredLength);

Why is StreamReader.ReadToEnd() able to read the entire request body successfully, while Stream.Read() can't? Reading the request stream twice feels like a hack. Is there a better way to go about this? (I only need to read the stream into a byte array once)

StreamReader uses a Windows File System method that closes the stream when the EOF is reached while Stream is not being closed by the far end. — jdweng, Oct 23 '19 at 13:30
["An implementation is free to return fewer bytes than requested even if the end of the stream has not been reached"](https://learn.microsoft.com/dotnet/api/system.io.stream.read). — Jeroen Mostert, Oct 23 '19 at 13:39
You should definitely look into switching over to [Pipelines](https://devblogs.microsoft.com/dotnet/system-io-pipelines-high-performance-io-in-net/). — Tanveer Badar, Oct 23 '19 at 13:45

canton7 · Accepted Answer · 2019-10-23T13:42:21.443

Remember that you're trying to read request.Body before all of the request has been received yet.

Stream.Read behaves like this:

If the end of the stream has been reached, return 0
If there are no bytes available which haven't already been read, block until at least 1 byte is available
If 1 or more new bytes are available, return them straight away. Don't block.

As you can see, if the whole body hasn't been received yet, request.Body.Read(...) will just return the part of the body that has been received.

StreamReader.ReadToEnd() calls Stream.Read in a loop, until it finds the end of the stream.

You should probably call Stream.Read in a loop as well, until you've read all of the bytes:

byte[] fileBuffer = new byte[declaredLength];
int numBytesRead = 0;
while (numBytesRead < declaredLength)
{
    int readBytes = request.Body.Read(fileBuffer, numBytesRead, declaredLength - numBytesRead);
    if (readBytes == 0)
    {
        // We reached the end of the stream before we were expecting it
        // Might want to throw an exception here?
    }
    numBytesRead += readBytes;
}

Why does StreamReader.ReadToEnd work but not Stream.Read?

1 Answers1