Using JSON.NET to serialize object into HttpClient's response stream

Question

Abstract

Hi, I'm working on a project where it is needed to send potentially huge json of some object via HttpClient, a 10-20 mb of JSON is a typical size. In order do that efficiently I want to use streams, both with Json.Net to serialize an object plus streams for posting data with HttpClient.

Problem

Here is the snippet for serialization with Json.net, in order to work with streams, Json.net expects a stream that it will write into:

public static void Serialize( object value, Stream writeOnlyStream )
{
    StreamWriter writer = new StreamWriter(writeOnlyStream);  <-- Here Json.net expects the stream to be already created
    JsonTextWriter jsonWriter = new JsonTextWriter(writer);
    JsonSerializer ser = new JsonSerializer();
    ser.Serialize(jsonWriter, value );
    jsonWriter.Flush();
}

While HttpClient expects a stream that it will read from:

using (var client = new HttpClient())
{
    client.BaseAddress = new Uri("http://localhost:54359/");

    var response = await client.PostAsync("/api/snapshot", new StreamContent(readOnlyStream)); <-- The same thing here, HttpClient expects the stream already to exist

    ... 
}

So eventually this means that both classes expecting the Stream to be created by someone else, but there are no streams both for Json.Net, neither for HttpClient. So the problem seems that can be solved by implementing a stream that would intercept a read requests made to read-only stream, and issue writes upon request from write-only stream.

Question

Maybe someone has stumbled on such situation already, and probably found already implemented solution to this problem. If so, please share it with me,

Thank you in advance!

score 15 · Answer 1 · answered Aug 16 '14 at 01:27

If you define a subclass of HttpContent :

public class JsonContent:HttpContent
{
    public object SerializationTarget{get;private set;}
    public JsonContent(object serializationTarget)
    {
        SerializationTarget=serializationTarget;
        this.Headers.ContentType = new MediaTypeHeaderValue("application/json");
    }
    protected override async Task SerializeToStreamAsync(Stream stream, 
                                                TransportContext context)
    {
        using(StreamWriter writer = new StreamWriter(stream))
        using(JsonTextWriter jsonWriter = new JsonTextWriter(writer))
        {
            JsonSerializer ser = new JsonSerializer();
            ser.Serialize(jsonWriter, SerializationTarget );
        }

    }   

    protected override bool TryComputeLength(out long length)
    {
        //we don't know. can't be computed up-front
        length = -1;
        return false;
    }
}

then you can:

var someObj = new {a = 1, b = 2};
var client = new HttpClient();
var content = new JsonContent(someObj);
var responseMsg = await client.PostAsync("http://someurl",content);

and the serializer will write directly to the request stream.

You have nice answer, thank you but I've used the one above because it's already existing implementation... — Lu4, Aug 19 '14 at 13:53
Great answer, for .Net Core 2.1 and above you will need to set leaveOpen to true to leave the Stream open after the StreamWriter is disposed. using (var writer = new StreamWriter(stream, new UTF8Encoding(false), 1024, true)) — DalSoft, Sep 19 '19 at 21:37
Any thoughts on how the above changes with regard to the System.Text.Json serializer in .Net Core 3.1? — Shawn de Wet, Mar 16 '20 at 14:35

score 5 · Accepted Answer · answered Aug 16 '14 at 01:55

Use PushStreamContent. Rather than have Web API "pull" from a stream, it lets you more intuitively "push" into one.

object value = ...;

PushStreamContent content = new PushStreamContent((stream, httpContent, transportContext) =>
{
    using (var tw = new StreamWriter(stream))
    {
        JsonSerializer ser = new JsonSerializer();
        ser.Serialize(tw, value);
    }
});

Note that JSON.NET doesn't support async during serialization so while this may be more memory efficient, it won't be very scalable.

I'd recommend trying to avoid such large JSON objects, though. Try to chunk it up, for instance, if you're sending over a large collection. Many clients/servers will flat out reject something so big without special handling.

awesome, but it seems to not be there in the httpClient nuget package in dotnet core? — Joel Harkes, Mar 08 '17 at 22:24

Using JSON.NET to serialize object into HttpClient's response stream

Abstract

Problem

Question

2 Answers2

Linked