Reading the stream then deserialising

Question

/// <summary> Deserialize Json streams </summary>
/// <param name="response"> The message we got to deserialize </param>
/// <param name="cancellationToken"> Cancellation settings depending on request </param>
/// <typeparam name="TResult"> Generic parameter </typeparam>
/// <returns> The <see cref="Task" />. we return the task </returns>
private static async Task<TResult> DeserializeAsync<TResult>(
            HttpResponseMessage response,
            CancellationToken cancellationToken)
{
     await using var contentStream = await response.Content.ReadAsStreamAsync().ConfigureAwait(false);

     #if DEBUG
     using var reader = new StreamReader(contentStream);
     var text = await reader.ReadToEndAsync().ConfigureAwait(false);
     Debug.WriteLine("RECEIVED: " + text);
     #endif

     return await JsonSerializer.DeserializeAsync<TResult>(
                contentStream,
                _serializerOptions,
                cancellationToken).ConfigureAwait(false);
}

Pretty simple code, it deserialises http response. But it crashes on the return. Both sets works on their own, return await deserialises async properly if I use it in release mode, but in debug mode the return wait fails because I believe the reader alters the stream. How do I fix this?

Should I make a copy of the stream and store that?

I've tried resetting the contentStream back to 0 but that just crashed the program instead.

Edit 3: Attempt 3:At @Magnus result Final fully working Implementation: Credit to User @Magnus

     private static async Task<TResult> DeserializeAsync<TResult>(
            HttpResponseMessage response,
            CancellationToken cancellationToken)
        {
            await using var contentStream = await GetStream(response).ConfigureAwait(false);

#if DEBUG
            contentStream.Position = 0;
            var reader = new StreamReader(contentStream);
            var text = await reader.ReadToEndAsync().ConfigureAwait(false);
            contentStream.Position = 0;
            Debug.WriteLine("RECEIVED: " + text);
#endif

            return await JsonSerializer.DeserializeAsync<TResult>(
                contentStream,
                _serializerOptions,
                cancellationToken).ConfigureAwait(false);
        }

        private static async Task<Stream> GetStream(HttpResponseMessage response)
        {
            var stream = await response.Content.ReadAsStreamAsync().ConfigureAwait(false);
#if DEBUG
            var target = new MemoryStream();
            await stream.CopyToAsync(target).ConfigureAwait(false);
            return target;
#endif
            return stream;
        }

Are you getting any exceptions? Make sure you verify you have data in the response and the data matches TResult type. I would check the status of the response to make sure you have a 200 OK and not an HTTP error. — jdweng, Jun 15 '20 at 08:35
The exception is not HTTP related. It's due to reading the stream twice. — JsonDork, Jun 15 '20 at 08:36
https://stackoverflow.com/questions/147941/how-can-i-read-an-http-response-stream-twice-in-c — Stewart Ritchie, Jun 15 '20 at 08:37
Why are you reading to end, just pass the reader to deserialize, I'm guessing you want a copy of the results, either make your own stream to do this, or buffer it to a memory stream or file stream first — TheGeneral, Jun 15 '20 at 08:37
@StewartRitchie That post does not seem relevant for this case. Same solution I don't think applies to this — JsonDork, Jun 15 '20 at 08:39
So some streams cannot be read more than once as your code attempts to do - for instance a stream coming from a socket (e.g. another server) - trying to set position to 0 will give you an exception as you have seen - there's no way to tell the other server to send the content again - you should copy into a MemoryStream during the first read. — Stewart Ritchie, Jun 15 '20 at 08:47
You are probably getting http 1.1 chunk mode (not 1.0 stream mode). So you have to put each chunk in the a buffer and then wait until you get all the chunks before processing. In chunk mode you get a 100 continue and the final chunk will have 200 ok. — jdweng, Jun 15 '20 at 09:23

Magnus · Accepted Answer · 2020-06-15T09:39:21.153

2

The problem is that the response stream does not support seek. So once you have read it you can not read it again. I would probably go deserializing the object into a variable and then make a function GetDebugInfo to generate a string from it. (Possibly using refelction since you dont know the type)

var obj = await JsonSerializer.DeserializeAsync<TResult>(
                contentStream,
                _serializerOptions,
                cancellationToken).ConfigureAwait(false);
#if DEBUG
  Debug.WriteLine("RECEIVED: " + GetDebugInfo(obj));
#endif

return obj

Another option would be to copy the Response stream to a memory stream if in debug mode.

private async Task<Stream> GetStream(HttpResponseMessage response)
{
  var stream = await response.Content.ReadAsStreamAsync();
#if DEBUG
  var target = new MemoryStream();
  await stream.CopyToAsync(target);
  target.Position = 0;
  return target;
#endif

   return stream;
}

private static async Task<TResult> DeserializeAsync<TResult>(
            HttpResponseMessage response,
            CancellationToken cancellationToken)
{
     await using var contentStream = await GetStream(response).ConfigureAwait(false);

     #if DEBUG
     var reader = new StreamReader(contentStream);
     var text = await reader.ReadToEndAsync().ConfigureAwait(false);
     contentStream.Position = 0;
     Debug.WriteLine("RECEIVED: " + text);
     #endif

     return await JsonSerializer.DeserializeAsync<TResult>(
                contentStream,
                _serializerOptions,
                cancellationToken).ConfigureAwait(false);
}

disposing the StreamReader will close the stream so remove the using.

edited Jun 15 '20 at 09:39

answered Jun 15 '20 at 08:40

Magnus

45,362
8
80
118

But this solution won't write the raw json correct? The reason for the debug statement is to send the text for debugging later on in case the models ever change from the back end team and thus making the deserialising fail. – JsonDork Jun 15 '20 at 08:44
Ok I see. I've added an alternative solution. – Magnus Jun 15 '20 at 08:48
Sadly I cannot get this one to work with my debug.writeline. – JsonDork Jun 15 '20 at 09:25
If you give more info what the problem is I can help you. – Magnus Jun 15 '20 at 09:31
Added my attempt at implementing this solution but nothing changed, but it works the excatly the same. Still the same exception trying to read the json, but still working in release mode. I.e still successfully deserialising once I skip the Whole compiler IF statement. – JsonDork Jun 15 '20 at 09:35
That is not how my solution looked. Keep your code as it was before, call GetStream, after you have written to the debug, set the position to 0. – Magnus Jun 15 '20 at 09:36
Oooh, almost works now! Now the text response is empty but I think I can figure that one out after some trial and error. But the deserialise works for debug now – JsonDork Jun 15 '20 at 09:46
Cheers Magnus! I got it working. Editted in the final implementation, I had to change where the position was resetted. Not sure why, but if I did set it to 0 inside the GetStream method it did not work, but closer to the use case. Probably some using error? – JsonDork Jun 15 '20 at 10:03

Reading the stream then deserialising

1 Answers1