How to deserialize JSON text on the fly in C#

Question

I am reading/receiving blocks of bytes from an async function which I buffer in a MemoryStream and then deserialize to a C# object.

    public async Task<Message> Read()
    {
        byte[] block = new byte[1024];
        int bytesRead;

        using var ms = new MemoryStream();
        while ((bytesRead = await GetBlock(block)) > 0)
        {
            ms.Write(block, 0, bytesRead);
        }
        ms.Seek(0, SeekOrigin.Begin);
        var result = await System.Text.Json.JsonSerializer.DeserializeAsync<Message>(ms);
        return result;
    }

I would prefer a method that would not buffer the whole data and keep a potentially large byte array in memory. Like a Dummy-Stream that forwards each block of bytes to the Desirailize function as it is beeing received by the GetBlocks function.

    public async Task<Message> Read()
    {
        byte[] block = new byte[1024];
        int bytesRead;

        using var rs = new RedirectStream((rs) =>
        {
            while ((bytesRead = await GetBlock(block)) > 0)
            {
                rs.Write(block, 0, bytesRead);
            }
        });

        var result = await System.Text.Json.JsonSerializer.DeserializeAsync<Message>(rs);
        return result;
    }

Does such a stream class exist or would I need to create my own (and if yes - where would I start?).

In addition to that, it would be interesting to know if the DeserializeAsync function would even be able to make good use of such a mechanism (meaning not reading the stream to the end before actually starting to deserialize it).

Answer 1

Does such a stream class exist

To my knowledge, there is nothing similar to your RedirectStream in the standard library, so you may have to implement it yourself, like other folks already did. A search for a NuGet package doesn't give pertinent results (like https://github.com/MarcusChr/BlockBasedMemoryStream where the removeReadData is misplaced and can't be used in your case).

[Does] DeserializeAsync function would even be able to make good use of such a mechanism

Actual version of JsonSerializer.DeserializeAsync use the Stream.ReadAsync method as you can see from the sources. So yes, for now (it's unlikely there is a regression).