55

I need to deserialize a complex JSON blob into standard .NET containers for use in code that is not aware of JSON. It expects things to be in standard .NET types, specifically Dictionary<string, object> or List<object> where "object" can be primitive or recurse (Dictionary or List).

I cannot use a static type to map the results and JObject/JToken don't fit. Ideally, there would be some way (via Contracts perhaps?) to convert raw JSON into basic .NET containers.

I've search all over for any way to coax the JSON.NET deserializer into creating these simple types when it encounters "{}" or "[]" but with little success.

Any help appreciated!

Brian Rogers
  • 125,747
  • 31
  • 299
  • 300
dongryphon
  • 705
  • 1
  • 5
  • 8
  • I tried System.Web.Script.Serialization.JavaScriptSerializer and it does what I want in this case, but I have other reasons for wanting to stick with JSON.NET. – dongryphon Apr 05 '11 at 01:38
  • Update: what I have done for now is to download and modify the source of Json.NET in the CreateJObject and CreateJToken methods to create the types I wanted. There were 8-10 unit tests to repair, but I can live with the resulting compromises. – dongryphon Apr 06 '11 at 01:05
  • For what it's worth, the issue stems from the users of the HasDefinedType method in JsonSerializerInternalReader. The HasDefinedType check is made **prior** to consulting a contract on how to create the target object and even if it did try that, the decision was already made as to the contract prior to knowing if a "{}" or "[]" was in play. I think there is some refactoring in order for Json.NET to externalize this decision and allow user code to determine the target type when "object" is all that is known. – dongryphon Apr 06 '11 at 01:10
  • 1
    Why is @brian-rogers answer not accepted as best answer? – Ignacio Calvo Jan 21 '16 at 11:17
  • @IgnacioCalvo: Because the question was clearly asked for an older version of Netwonsoft.Json that couldn't do it yet. – Joshua Sep 13 '19 at 17:24

5 Answers5

74

If you just want a generic method that can handle any arbitrary JSON and convert it into a nested structure of regular .NET types (primitives, Lists and Dictionaries), you can use JSON.Net's LINQ-to-JSON API to do it:

using System.Linq;
using Newtonsoft.Json.Linq;

public static class JsonHelper
{
    public static object Deserialize(string json)
    {
        return ToObject(JToken.Parse(json));
    }

    public static object ToObject(JToken token)
    {
        switch (token.Type)
        {
            case JTokenType.Object:
                return token.Children<JProperty>()
                            .ToDictionary(prop => prop.Name,
                                          prop => ToObject(prop.Value));

            case JTokenType.Array:
                return token.Select(ToObject).ToList();

            default:
                return ((JValue)token).Value;
        }
    }
}

You can call the method as shown below. obj will either contain a Dictionary<string, object>, List<object>, or primitive depending on what JSON you started with.

object obj = JsonHelper.Deserialize(jsonString);
Brian Rogers
  • 125,747
  • 31
  • 299
  • 300
18

One way to deserialize a json string recursively into dictionaries and lists with JSON.NET is to create a custom json converter class that derives from the JsonConverter abstract class provided by JSON.NET.

It is in your derived JsonConverter where you put the implementation of how an object should be written to and from json.

You can use your custom JsonConverter like this:

var o = JsonConvert.DeserializeObject<IDictionary<string, object>>(json, new DictionaryConverter());

Here is a custom JsonConverter I have used with success in the past to achieve the same goals as you outline in your question:

public class DictionaryConverter : JsonConverter {
    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) { this.WriteValue(writer, value); }

    private void WriteValue(JsonWriter writer, object value) {
        var t = JToken.FromObject(value);
        switch (t.Type) {
            case JTokenType.Object:
                this.WriteObject(writer, value);
                break;
            case JTokenType.Array:
                this.WriteArray(writer, value);
                break;
            default:
                writer.WriteValue(value);
                break;
        }
    }

    private void WriteObject(JsonWriter writer, object value) {
        writer.WriteStartObject();
        var obj = value as IDictionary<string, object>;
        foreach (var kvp in obj) {
            writer.WritePropertyName(kvp.Key);
            this.WriteValue(writer, kvp.Value);
        }
        writer.WriteEndObject();
    }

    private void WriteArray(JsonWriter writer, object value) {
        writer.WriteStartArray();
        var array = value as IEnumerable<object>;
        foreach (var o in array) {
            this.WriteValue(writer, o);
        }
        writer.WriteEndArray();
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) {
        return ReadValue(reader);
    }

    private object ReadValue(JsonReader reader) {
        while (reader.TokenType == JsonToken.Comment) {
            if (!reader.Read()) throw new JsonSerializationException("Unexpected Token when converting IDictionary<string, object>");
        }

        switch (reader.TokenType) {
            case JsonToken.StartObject:
                return ReadObject(reader);
            case JsonToken.StartArray:
                return this.ReadArray(reader);
            case JsonToken.Integer:
            case JsonToken.Float:
            case JsonToken.String:
            case JsonToken.Boolean:
            case JsonToken.Undefined:
            case JsonToken.Null:
            case JsonToken.Date:
            case JsonToken.Bytes:
                return reader.Value;
            default:
                throw new JsonSerializationException
                    (string.Format("Unexpected token when converting IDictionary<string, object>: {0}", reader.TokenType));
        }
    }

    private object ReadArray(JsonReader reader) {
        IList<object> list = new List<object>();

        while (reader.Read()) {
            switch (reader.TokenType) {
                case JsonToken.Comment:
                    break;
                default:
                    var v = ReadValue(reader);

                    list.Add(v);
                    break;
                case JsonToken.EndArray:
                    return list;
            }
        }

        throw new JsonSerializationException("Unexpected end when reading IDictionary<string, object>");
    }

    private object ReadObject(JsonReader reader) {
        var obj = new Dictionary<string, object>();

        while (reader.Read()) {
            switch (reader.TokenType) {
                case JsonToken.PropertyName:
                    var propertyName = reader.Value.ToString();

                    if (!reader.Read()) {
                        throw new JsonSerializationException("Unexpected end when reading IDictionary<string, object>");
                    }

                    var v = ReadValue(reader);

                    obj[propertyName] = v;
                    break;
                case JsonToken.Comment:
                    break;
                case JsonToken.EndObject:
                    return obj;
            }
        }

        throw new JsonSerializationException("Unexpected end when reading IDictionary<string, object>");
    }

    public override bool CanConvert(Type objectType) { return typeof(IDictionary<string, object>).IsAssignableFrom(objectType); }
}

Here is the equivalent in f#:

type IDictionaryConverter() =
    inherit JsonConverter()

    let rec writeValue (writer: JsonWriter) (value: obj) =
            let t = JToken.FromObject(value)
            match t.Type with
            | JTokenType.Object -> writeObject writer value
            | JTokenType.Array -> writeArray writer value
            | _ -> writer.WriteValue value    

    and writeObject (writer: JsonWriter) (value: obj) =
        writer.WriteStartObject ()
        let obj = value :?> IDictionary<string, obj>
        for kvp in obj do
            writer.WritePropertyName kvp.Key
            writeValue writer kvp.Value
        writer.WriteEndObject ()    

    and writeArray (writer: JsonWriter) (value: obj) = 
        writer.WriteStartArray ()
        let array = value :?> IEnumerable<obj>
        for o in array do
            writeValue writer o
        writer.WriteEndArray ()

    let rec readValue (reader: JsonReader) =
        while reader.TokenType = JsonToken.Comment do
            if reader.Read () |> not then raise (JsonSerializationException("Unexpected token when reading object"))

        match reader.TokenType with
        | JsonToken.Integer
        | JsonToken.Float
        | JsonToken.String
        | JsonToken.Boolean
        | JsonToken.Undefined
        | JsonToken.Null
        | JsonToken.Date
        | JsonToken.Bytes -> reader.Value
        | JsonToken.StartObject -> readObject reader Map.empty
        | JsonToken.StartArray -> readArray reader []
        | _ -> raise (JsonSerializationException(sprintf "Unexpected token when reading object: %O" reader.TokenType))


    and readObject (reader: JsonReader) (obj: Map<string, obj>) =
        match reader.Read() with
        | false -> raise (JsonSerializationException("Unexpected end when reading object"))
        | _ -> reader.TokenType |> function
            | JsonToken.Comment -> readObject reader obj
            | JsonToken.PropertyName ->
                let propertyName = reader.Value.ToString ()
                if reader.Read() |> not then raise (JsonSerializationException("Unexpected end when reading object"))
                let value = readValue reader
                readObject reader (obj.Add(propertyName, value))
            | JsonToken.EndObject -> box obj
            | _ -> raise (JsonSerializationException(sprintf "Unexpected token when reading object: %O" reader.TokenType))

    and readArray (reader: JsonReader) (collection: obj list) =
        match reader.Read() with
        | false -> raise (JsonSerializationException("Unexpected end when reading array"))
        | _ -> reader.TokenType |> function
            | JsonToken.Comment -> readArray reader collection
            | JsonToken.EndArray -> box collection
            | _ -> collection @ [readValue reader] |> readArray reader

    override __.CanConvert t = (typeof<IDictionary<string, obj>>).IsAssignableFrom t
    override __.WriteJson (writer:JsonWriter, value: obj, _:JsonSerializer) = writeValue writer value 
    override __.ReadJson (reader:JsonReader, _: Type, _:obj, _:JsonSerializer) = readValue reader
Anish Patel
  • 4,332
  • 1
  • 30
  • 45
  • Thanks for that. It resolved half of the challenge. Second half is to use not C# objects but MatLab MWArray derived objects. Here I have access to each value and it's property name, so it seems like straight road from here. – Mariusz Jan 12 '17 at 15:33
1

I love AutoMapper and seem to think it solves many problems... like this one...

why not just let the JSON.NET convert the thing into whatever it wants to... and use AutoMapper to map it into the object you really want.

Unless performance is paramount this extra step should be worth it for the reduction in complexity and the ability to use the serializer you want.

Tim Sylvester
  • 22,897
  • 2
  • 80
  • 94
John Sobolewski
  • 4,512
  • 1
  • 20
  • 26
  • Thanks I'll look into that. But I still hope to find something native to JSON.NET since it has to create something when confronted with "{}" to "object" or "[]" to object. I just cannot see how to control the type of object it creates in this case. – dongryphon Apr 05 '11 at 15:10
0

You can have full control over the serialization of a type by using a custom JsonConverter. Documentation at http://james.newtonking.com/projects/json/help/html/T_Newtonsoft_Json_JsonConverter.htm .

Also, according to this blog post you need to use JArray for a List, and JObject for a dictionary.

smartcaveman
  • 41,281
  • 29
  • 127
  • 212
  • Thanks for tips. I am needing to handle deserialization of objects based on the JSON: "{}" needs to create a Dictionary[string, object] and "[]" needs to create a List[object] or plain object[]. I don't see how to connect JsonCoverter to this problem. There appears to be some hard coded logic in the deserializer even before it uses the contracts when the target type is "object". – dongryphon Apr 05 '11 at 15:07
  • Override the contract resolver to wire up the the custom converter – smartcaveman Apr 05 '11 at 15:08
  • Thanks, but I have tried that. The contracts are not used when the type falls into the "!HasDefinedType" check in the deserializer. Take a peak at JsonSerializerInternalReader.cs and search for HasDefinedType. You'll see a call to this method just above the delegation to a contract and if the type is "object", it gets caught by this check. – dongryphon Apr 08 '11 at 01:38
0

You cannot do what I was asking. At least not as far as I can tell after MUCH research. I had to edit the source of Json.NET.

Robert Harvey
  • 178,213
  • 47
  • 333
  • 501
dongryphon
  • 705
  • 1
  • 5
  • 8
  • any chance this change was pushed back into the source or available anywhere? – Maslow Jul 06 '11 at 01:14
  • 1
    I have a similar problem where my dictionary values sometimes contain arrays (basically "[]"). I've hit this problem for the first time, but struggle to understand why this hasn't been solved in a generic way already. This is a dead end on what seems to be a pretty basic problem. Anyone want to chime in and explain what the major issue for JSON deserialization (JSON.NET) is? Does everyone else have control over their JSON and structure it "better" or what are we missing here? – PandaWood Jun 14 '12 at 00:05