System.Text.Json.JsonElement ToObject workaround

Question

I want to know the equivalent of the ToObject<>() method in Json.NET for System.Text.Json.

Using Json.NET you can use any JToken and convert it to a class. For example:

var str = ""; // Some JSON string
var jObj = JObject.Parse(str);
var myClass = jObj["SomeProperty"].ToObject<SomeClass>();

How would we be able to do this with .NET Core 3's new System.Text.Json?

var str = ""; // Some JSON string
var jDoc = JsonDocument.Parse(str);
var myClass = jDoc.RootElement.GetProperty("SomeProperty"). <-- now what??

Initially, I was thinking I'd just convert the JsonElement that is returned in jDoc.RootElement.GetPRoperty("SomeProperty") to a string and then deserialize that string. But I feel that might not be the most efficient method, and I can't really find documentation on doing it another way.

score 115 · Answer 1 · edited Aug 10 '22 at 14:45

In .NET 6, extension methods are being added to JsonSerializer to deserialize an object directly from a JsonElement or JsonDocument:

public static partial class JsonSerializer
{
    public static TValue? Deserialize<TValue>(this JsonDocument document, JsonSerializerOptions? options = null);
    public static object? Deserialize(this JsonDocument document, Type returnType, JsonSerializerOptions? options = null);
    public static TValue? Deserialize<TValue>(this JsonDocument document, JsonTypeInfo<TValue> jsonTypeInfo);
    public static object? Deserialize(this JsonDocument document, Type returnType, JsonSerializerContext context);

    public static TValue? Deserialize<TValue>(this JsonElement element, JsonSerializerOptions? options = null);
    public static object? Deserialize(this JsonElement element, Type returnType, JsonSerializerOptions? options = null);
    public static TValue? Deserialize<TValue>(this JsonElement element, JsonTypeInfo<TValue> jsonTypeInfo);
    public static object? Deserialize(this JsonElement element, Type returnType, JsonSerializerContext context);
}

Now you will be able to do:

using var jDoc = JsonDocument.Parse(str);
var myClass = jDoc.RootElement.GetProperty("SomeProperty").Deserialize<SomeClass>();

Notes:

JsonDocument is disposable. According to the documentation, This class utilizes resources from pooled memory... failure to properly dispose this object will result in the memory not being returned to the pool, which will increase GC impact across various parts of the framework.

So, be sure to declare your jDoc with a using statement.
The new methods should be present in .NET 6.0 Preview RC1.

They were added in response to the enhancement request We should be able serialize and serialize from DOM #31274, which has been closed.

Similar extension methods were added for the new JsonNode mutable JSON document node as well

public static TValue? Deserialize<TValue>(this JsonNode? node, JsonSerializerOptions? options = null)
public static object? Deserialize(this JsonNode? node, Type returnType, JsonSerializerOptions? options = null)
public static TValue? Deserialize<TValue>(this JsonNode? node, JsonTypeInfo<TValue> jsonTypeInfo)
public static object? Deserialize(this JsonNode? node, Type returnType, JsonSerializerContext context)

In .NET 5 and earlier these methods do not exist. As a workaround, you may get better performance by writing to an intermediate byte buffer rather than to a string, since both JsonDocument and Utf8JsonReader work directly with byte spans rather than strings or char spans. As stated in the documentation:

Serializing to UTF-8 is about 5-10% faster than using the string-based methods. The difference is because the bytes (as UTF-8) don't need to be converted to strings (UTF-16).

public static partial class JsonExtensions
{
    public static T ToObject<T>(this JsonElement element, JsonSerializerOptions options = null)
    {
        var bufferWriter = new ArrayBufferWriter<byte>();
        using (var writer = new Utf8JsonWriter(bufferWriter))
            element.WriteTo(writer);
        return JsonSerializer.Deserialize<T>(bufferWriter.WrittenSpan, options);
    }

    public static T ToObject<T>(this JsonDocument document, JsonSerializerOptions options = null)
    {
        if (document == null)
            throw new ArgumentNullException(nameof(document));
        return document.RootElement.ToObject<T>(options);
    }
}

A demo fiddle is here.

Is it safe to `Deserialize()` using `options` if these options are not used when writing the internal buffer? — stb, Feb 19 '21 at 16:53
Still using this answer on dotnet 5 some 18 months later. Needs a ```writer.Flush();``` after ```element.WriteTo(writer);``` though otherwise you sometimes get ```System.Text.Json.JsonException: 'Expected depth to be zero at the end of the JSON payload. There is an open JSON object or array that should be closed.``` — mclayton, Mar 03 '21 at 21:47
@mclayton - I do dispose of the `Utf8JsonWriter writer` via a `using` statement, and according to the docs the [dispose method](https://learn.microsoft.com/en-us/dotnet/api/system.text.json.utf8jsonwriter.dispose) *In the case of IBufferWriter, this advances the underlying `IBufferWriter` based on what has been written so far.* and the [flush method](https://learn.microsoft.com/en-us/dotnet/api/system.text.json.utf8jsonwriter.flush) also *advances the underlying `IBufferWriter` based on what has been written so far.* Doing both does not seem necessary. — dbc, Mar 05 '21 at 18:07
@mclayton - Thus it is surprising that `writer.Flush()` is required as well as the `using` statement. can you share a [mcve] showing a case where `Flush()` and `Dispose()` are both required? — dbc, Mar 05 '21 at 18:07
Ah, my bad. I was “using” the C# 8.0 form of using ... ```using var writer = new Utf8JsonWriter(bufferWriter);``` so the ```Dispose``` only gets called at the end of the function, which totally changes the behaviour of the code and needed the ```Flush``` to fix it. What a difference a couple of brackets makes! — mclayton, Mar 05 '21 at 18:27
If the `JsonElement` holds a smaller json string (length wise), will it still be better to do like you suggest, or could the cost of setting up the writers be less performant compared to using `GetRawText()`? ... Or set the Deserialize method up such writer itself, if it gets a `string`? — Asons, May 29 '21 at 08:44
@Ason - you can check the source. Internally the contents of `JsonElement` are stored as a `ReadOnlyMemory segment` utf8 byte sequence, so [`GetRawText()`](https://github.com/dotnet/runtime/blob/v5.0.6/src/libraries/System.Text.Json/src/System/Text/Json/Document/JsonElement.cs#L1165) => [`GetRawValueAsString(int index)`](https://github.com/dotnet/runtime/blob/v5.0.6/src/libraries/System.Text.Json/src/System/Text/Json/Document/JsonDocument.cs#L756) transcodes that to a new string before returning it. — dbc, May 29 '21 at 16:13
@Ason - And `JsonSerializer` internally deserializes from utf8 byte sequences, so [`JsonSerializer.Deserialize(string json, ...)`](https://github.com/dotnet/runtime/blob/v5.0.6/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.String.cs#L86) transcodes back to utf8 before deserializing. My methods simply skip the transcoding back and forth. — dbc, May 29 '21 at 16:15
`ArrayBufferWriter` seems to be a .NET Core thing. Can I replace it with `ArrayPoolBufferWriter` from `Microsoft.Toolkit.HighPerformance` nuget package when working in .NET Framework 4.8? — dotNET, Feb 04 '22 at 10:47
@dotNET - Sure. You can use anything that implements `IBufferWriter`. It seems to work fine, see https://dotnetfiddle.net/SdwAUt, and it might result in better performance if you do so. Be sure to dispose it afterwards e.g. `using var bufferWriter = new ArrayPoolBufferWriter();`. — dbc, Feb 04 '22 at 15:10

score 106 · Answer 2 · answered Oct 01 '19 at 22:40

106

I came across the same issue, so I wrote some extension methods which work fine for now. It would be nice if they provided this as built in to avoid the additional allocation to a string.

public static T ToObject<T>(this JsonElement element)
{
    var json = element.GetRawText();
    return JsonSerializer.Deserialize<T>(json);
}
public static T ToObject<T>(this JsonDocument document)
{
    var json = document.RootElement.GetRawText();
    return JsonSerializer.Deserialize<T>(json);
}

Then use as follows:

jDoc.RootElement.GetProperty("SomeProperty").ToObject<SomeClass>();

answered Oct 01 '19 at 22:40

Gary Holland

2,565
1
16
17

4

This is exactly what I ended up doing as well. I posted the question to see if there was a more efficient way without that string allocation as you have mentioned. Maybe we'll have to wait until system.text.json is a little more mature. – Stalfos Oct 02 '19 at 17:57
14

Yup, though it isn't about string allocation only. It's more about parsing the textual representation of the whole JSON again. It was already parsed once when the JsonElement/Document was created so it's a waste of CPU too. – Elephantik May 21 '20 at 14:46
1

dude... EXACTLY WHAT I Was trying to do. Thanks so much for adding this! – ganjeii Feb 08 '22 at 22:19

Paul · Answer 3 · 2021-02-20T01:52:33.933

Same as dbc's answer, just including the methods which allow you to specify a return type via Type returnType.

public static partial class JsonExtensions
{
    public static T ToObject<T>(this JsonElement element, JsonSerializerOptions options = null)
    {
        var bufferWriter = new ArrayBufferWriter<byte>();
        using (var writer = new Utf8JsonWriter(bufferWriter))
        {
            element.WriteTo(writer);
        }

        return JsonSerializer.Deserialize<T>(bufferWriter.WrittenSpan, options);
    }

    public static T ToObject<T>(this JsonDocument document, JsonSerializerOptions options = null)
    {
        if (document == null)
        {
            throw new ArgumentNullException(nameof(document));
        }

        return document.RootElement.ToObject<T>(options);
    }       

    public static object ToObject(this JsonElement element, Type returnType, JsonSerializerOptions options = null)
    {
        var bufferWriter = new ArrayBufferWriter<byte>();
        using (var writer = new Utf8JsonWriter(bufferWriter))
        {
            element.WriteTo(writer);
        }

        return JsonSerializer.Deserialize(bufferWriter.WrittenSpan, returnType, options);
    }

    public static object ToObject(this JsonDocument document, Type returnType, JsonSerializerOptions options = null)
    {
        if (document == null)
        {
            throw new ArgumentNullException(nameof(document));
        }

        return document.RootElement.ToObject(returnType, options);
    }       
}

serializing back to a text seems not a efficient way – Mightywill Mar 17 '21 at 09:31 — Mightywill, Mar 17 '21 at 09:31

score 7 · Answer 4 · answered Feb 18 '22 at 11:56

.NET 6 introduced the System.Text.Json.Nodes namespace, which provides a way to do this using almost exactly the same syntax as Json.Net:

var str = ""; // some json string
var node = JsonNode.Parse(str);
var myClass = node["SomeProperty"].Deserialize<SomeClass>();

The namespace includes 4 new types: JsonNode, JsonArray, JsonObject, and JsonValue which can be used to access or modify values within the DOM. JsonNode is the base class for the other three types.

The Deserialize extension methods listed in dbc's answer have also been added to operate on JsonNode, eg:

public static TValue? Deserialize<TValue>(this JsonNode? node, JsonSerializerOptions? options = null);

JsonNode is not disposable so you do not need to use the using syntax.

Use AsObject() or AsArray() to parse to a JsonObject or JsonArray, respectively:

// parse array
JsonArray arr = JsonNode.Parse(@"[{""Name"": ""Bob"", ""Age"":30}]").AsArray();
// parse object
JsonObject obj = JsonNode.Parse(@"{""Name"": ""Bob"", ""Age"":30}").AsObject();
// get a value
var date = JsonNode.Parse(@"{""Date"":""2021-12-21T13:24:46+04:00""}")["Date"].GetValue<DateTimeOffset>();

Once the json has been parsed it's possible to navigate, filter and transform the DOM and/or apply Deserialize<T>() to map to your concrete type.

To serialize back to a json string you can use ToJsonString(), eg:

string innerNodeJson = node["SomeProperty"].ToJsonString();

Please see this answer to Equivalent of JObject in System.Text.Json for more details information about JsonObject.

System.Text.Json.JsonElement ToObject workaround

4 Answers4

Linked

Related