10

NOTE: I am using Microsoft's new System.Text.Json and not Json.NET so make sure answers address this accordingly.

Consider these simple POCOs:

interface Vehicle {}

class Car : Vehicle {
    string make          { get; set; }
    int    numberOfDoors { get; set; }
}

class Bicycle : Vehicle {
    int frontGears { get; set; }
    int backGears  { get; set; }
}

The car can be represented in JSON like this...

{
  "make": "Smart",
  "numberOfDoors": 2
}

and the bicycle can be represented like this...

{
  "frontGears": 3,
  "backGears": 6
}

Pretty straight forward. Now consider this JSON.

[
  {
    "Car": {
      "make": "Smart",
      "numberOfDoors": 2
    }
  },
  {
    "Car": {
      "make": "Lexus",
      "numberOfDoors": 4
    }
  },
  {
    "Bicycle" : {
      "frontGears": 3,
      "backGears": 6
    }
  }
]

This is an array of objects where the property name is the key to know which type the corresponding nested object refers to.

While I know how to write a custom converter that uses the UTF8JsonReader to read the property names (e.g. 'Car' and 'Bicycle' and can write a switch statement accordingly, what I don't know is how to fall back to the default Car and Bicycle converters (i.e. the standard JSON converters) since I don't see any method on the reader to read in a specific typed object.

So how can you manually deserialize nested objects like this?

Mark A. Donohoe
  • 28,442
  • 25
  • 137
  • 286
  • A great question to challenge on :-) would AutoMapper be of any help on this? or are you looking for no-library based conversions/mapping? I'll take this as like an interview coding exercise... – Ak777 Jan 15 '20 at 00:08
  • 1
    No Library. BUT... I think I'm on to something here... be back in a minute! – Mark A. Donohoe Jan 15 '20 at 00:10
  • Does this answer your question? [Casting interfaces for deserialization in JSON.NET](https://stackoverflow.com/questions/5780888/casting-interfaces-for-deserialization-in-json-net) – Jawad Jan 15 '20 at 01:11
  • That looks like it's for NewtonSoft.Json. I'm trying to use System.Text.Json – Mark A. Donohoe Jan 15 '20 at 01:45

4 Answers4

10

I figured it out. You simply pass your reader/writer down to another instance of the JsonSerializer and it handles it as if it were a native object.

Here's a complete example you can paste into something like RoslynPad and just run it.

Here's the implementation...

using System;
using System.Collections.ObjectModel;
using System.Text.Json;
using System.Text.Json.Serialization;

public class HeterogenousListConverter<TItem, TList> : JsonConverter<TList>
where TItem : notnull
where TList : IList<TItem>, new() {

    public HeterogenousListConverter(params (string key, Type type)[] mappings){
        foreach(var (key, type) in mappings)
            KeyTypeLookup.Add(key, type);
    }

    public ReversibleLookup<string, Type> KeyTypeLookup = new ReversibleLookup<string, Type>();

    public override bool CanConvert(Type typeToConvert)
        => typeof(TList).IsAssignableFrom(typeToConvert);

    public override TList Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options){

        // Helper function for validating where you are in the JSON    
        void validateToken(Utf8JsonReader reader, JsonTokenType tokenType){
            if(reader.TokenType != tokenType)
                throw new JsonException($"Invalid token: Was expecting a '{tokenType}' token but received a '{reader.TokenType}' token");
        }

        validateToken(reader, JsonTokenType.StartArray);

        var results = new TList();

        reader.Read(); // Advance to the first object after the StartArray token. This should be either a StartObject token, or the EndArray token. Anything else is invalid.

        while(reader.TokenType == JsonTokenType.StartObject){ // Start of 'wrapper' object

            reader.Read(); // Move to property name
            validateToken(reader, JsonTokenType.PropertyName);

            var typeKey = reader.GetString();

            reader.Read(); // Move to start of object (stored in this property)
            validateToken(reader, JsonTokenType.StartObject); // Start of vehicle

            if(KeyTypeLookup.TryGetValue(typeKey, out var concreteItemType)){
                var item = (TItem)JsonSerializer.Deserialize(ref reader, concreteItemType, options);
                results.Add(item);
            }
            else{
                throw new JsonException($"Unknown type key '{typeKey}' found");
            }

            reader.Read(); // Move past end of item object
            reader.Read(); // Move past end of 'wrapper' object
        }

        validateToken(reader, JsonTokenType.EndArray);

        return results;
    }

    public override void Write(Utf8JsonWriter writer, TList items, JsonSerializerOptions options){

        writer.WriteStartArray();

        foreach (var item in items){

            var itemType = item.GetType();            

            writer.WriteStartObject();

            if(KeyTypeLookup.ReverseLookup.TryGetValue(itemType, out var typeKey)){
                writer.WritePropertyName(typeKey);
                JsonSerializer.Serialize(writer, item, itemType, options);
            }
            else{
                throw new JsonException($"Unknown type '{itemType.FullName}' found");
            }

            writer.WriteEndObject();
        }

        writer.WriteEndArray();
    }
}

Here's the demo code...

#nullable disable

public interface IVehicle { }

public class Car : IVehicle {
    public string make          { get; set; } = null;
    public int    numberOfDoors { get; set; } = 0;

    public override string ToString()
        => $"{make} with {numberOfDoors} doors";
}

public class Bicycle : IVehicle{
    public int frontGears { get; set; } = 0;
    public int backGears  { get; set; } = 0;

    public override string ToString()
        => $"{nameof(Bicycle)} with {frontGears * backGears} gears";
}

string json = @"[
  {
    ""Car"": {
      ""make"": ""Smart"",
      ""numberOfDoors"": 2
    }
  },
  {
    ""Car"": {
      ""make"": ""Lexus"",
      ""numberOfDoors"": 4
    }
  },
  {
    ""Bicycle"": {
      ""frontGears"": 3,
      ""backGears"": 6
    }
  }
]";

var converter = new HeterogenousListConverter<IVehicle, ObservableCollection<IVehicle>>(
    (nameof(Car),     typeof(Car)),
    (nameof(Bicycle), typeof(Bicycle))
);

var options = new JsonSerializerOptions();
options.Converters.Add(converter);

var vehicles = JsonSerializer.Deserialize<ObservableCollection<IVehicle>>(json, options);
Console.Write($"{vehicles.Count} Vehicles: {String.Join(", ",  vehicles.Select(v => v.ToString())) }");

var json2 = JsonSerializer.Serialize(vehicles, options);
Console.WriteLine(json2);

Console.WriteLine($"Completed at {DateTime.Now}");

Here's the supporting two-way lookup used above...

using System.Collections.ObjectModel;
using System.Diagnostics;

public class ReversibleLookup<T1, T2> : ReadOnlyDictionary<T1, T2>
where T1 : notnull 
where T2 : notnull {

    public ReversibleLookup(params (T1, T2)[] mappings)
    : base(new Dictionary<T1, T2>()){

        ReverseLookup = new ReadOnlyDictionary<T2, T1>(reverseLookup);

        foreach(var mapping in mappings)
            Add(mapping.Item1, mapping.Item2);
    }

    private readonly Dictionary<T2, T1> reverseLookup = new Dictionary<T2, T1>();
    public ReadOnlyDictionary<T2, T1> ReverseLookup { get; }

    [DebuggerHidden]
    public void Add(T1 value1, T2 value2) {

        if(ContainsKey(value1))
            throw new InvalidOperationException($"{nameof(value1)} is not unique");

        if(ReverseLookup.ContainsKey(value2))
            throw new InvalidOperationException($"{nameof(value2)} is not unique");

        Dictionary.Add(value1, value2);
        reverseLookup.Add(value2, value1);
    }

    public void Clear(){
        Dictionary.Clear();
        reverseLookup.Clear();        
    }
}
Mark A. Donohoe
  • 28,442
  • 25
  • 137
  • 286
  • 1
    https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-converters-how-to#support-polymorphic-deserialization This doc show a similar example for polymorphic deserialization but it goes and creates the types directly and uses the reader to fill the property values rather than calling the `JsonSerializer`. For more complicated types (or if you don't want such fine grain control), it makes sense to rely on the `JsonSerializer` once you know the type. – ahsonkhan Jan 15 '20 at 02:09
  • That's what I was trying to solve... nested concrete types. Their example is fine for simple objects, but the minute you get into other concrete classes, there's no reason to reinvent the wheel. I was looking for something to simply 'set-and-forget' and only worry about the key (property name) and the type the data is stored for. Everything else shouldn't be handled by me. – Mark A. Donohoe Jan 15 '20 at 02:15
  • 1
    Great solution, solves a lot of my problems. Thanks! – Clicktricity Oct 13 '20 at 17:10
  • ? i don't understand ReversibleLookup why does it inherit ReadOnlyDictionary, this makes no sense if it contains a `Dictionary reverseLookup` aka now you have two one which is a prop and one which come from base. why do you have this? surely one can be removed... i would remove the `ReadOnlyDictionary` you loose nothing and make more sense no point in having two dictionarys, @MarkA.Donohoe – Seabizkit Nov 02 '20 at 12:00
  • It's a standard pattern. Reversible lookup needs two directions: one way is handled by 'base' and the other is handled by the added property. This means you have hash-table lookups going in both directions. It also ensures no duplicate entries on either side. If you removed one, you would not only lose that protection, but you'd also have to manually search the 'values' side for a match to get the keys, which is a lot slower than hash-based lookups. – Mark A. Donohoe Nov 02 '20 at 17:58
  • 1
    Thanks a lot Mark for your answer. Your code is very reusable, and was useful to my project – Emmanuel DURIN Mar 04 '21 at 13:52
  • Glad I could help, @EmmanuelDURIN! :) – Mark A. Donohoe Nov 06 '21 at 18:43
  • @Clicktricity, glad it helped solve your problems! :) – Mark A. Donohoe Nov 06 '21 at 18:43
2

Here is another solution that builds upon the previous ones (with slightly different JSON structure).

Notable differences:

  • Discriminator is part of the object (no need to use wrapper objects)
  • To my own surprise, it is not necessary to remove the converter with recursive (de)serialize calls (.NET 6)
  • I didn't add custom lookup, see previous answers

The code:

var foo = new[] {
    new Foo
    {
        Inner = new Bar
        {
            Value = 42,
        },
    },
    new Foo
    {
        Inner = new Baz
        {
            Value = "Hello",
        },
    },
};

var opts = new JsonSerializerOptions
{
    Converters =
    {
        new PolymorphicJsonConverterWithDiscriminator<Base>(typeof(Bar), typeof(Baz)),
    },

};

var json = JsonSerializer.Serialize(foo, opts);
var foo2 = JsonSerializer.Deserialize<Foo[]>(json, opts);

Console.WriteLine(foo2 is not null && foo2.SequenceEqual(foo));
Console.ReadLine();
 
public static class Constants
{
    public const string DiscriminatorPropertyName = "$type";
}

public record Foo
{
    public Base? Inner { get; set; }
}

public abstract record Base();

public record Bar : Base
{
    [JsonPropertyName(DiscriminatorPropertyName)]
    [JsonPropertyOrder(int.MinValue)]
    public string TypeDiscriminator { get => nameof(Bar); init { if (value != nameof(Bar)) throw new ArgumentException(); } }
    public int Value { get; set; }
}

public record Baz : Base
{
    [JsonPropertyName(DiscriminatorPropertyName)]
    [JsonPropertyOrder(int.MinValue)]
    public string TypeDiscriminator { get => nameof(Baz); init { if (value != nameof(Baz)) throw new ArgumentException(); } }
    public string? Value { get; set; }
}

public class PolymorphicJsonConverterWithDiscriminator<TBase> : JsonConverter<TBase>
    where TBase : class
{
    private readonly Type[] supportedTypes;

    public PolymorphicJsonConverterWithDiscriminator(params Type[] supportedTypes)
    {
        this.supportedTypes = supportedTypes;
    }

    public override TBase? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        // Clone the reader so we can pass the original to Deserialize.
        var readerClone = reader;

        if (readerClone.TokenType == JsonTokenType.Null)
        {
            return null;
        }

        if (readerClone.TokenType != JsonTokenType.StartObject)
        {
            throw new JsonException();
        }

        readerClone.Read();
        if (readerClone.TokenType != JsonTokenType.PropertyName)
        {
            throw new JsonException();
        }

        var propertyName = readerClone.GetString();
        if (propertyName != DiscriminatorPropertyName)
        {
            throw new JsonException();
        }

        readerClone.Read();
        if (readerClone.TokenType != JsonTokenType.String)
        {
            throw new JsonException();
        }

        var typeIdentifier = readerClone.GetString();

        var specificType = supportedTypes.FirstOrDefault(t => t.Name == typeIdentifier)
            ?? throw new JsonException();

        return (TBase?)JsonSerializer.Deserialize(ref reader, specificType, options);
    }

    public override void Write(Utf8JsonWriter writer, TBase value, JsonSerializerOptions options)
    {
        // Cast to object which forces the serializer to use runtime type.
        JsonSerializer.Serialize(writer, value, typeof(object), options);
    }
}

Sample JSON:

[
  {
    "Inner": {
      "$type": "Bar",
      "Value": 42
    }
  },
  {
    "Inner": {
      "$type": "Baz",
      "Value": "Hello"
    }
  }
]
Honza R
  • 711
  • 7
  • 14
0

Here is a solution that works with single objects (does not require an array of objects). This is a copy of https://stackoverflow.com/a/59744873 modified to work without IList.

This is the main class

using System;
using System.Text.Json;
using System.Text.Json.Serialization;

namespace Shared.DataAccess
{
    /// <summary>
    /// Enables System.Text.Json to handle polymorphic classes
    /// The polymorphic classes must be explicitly mapped
    /// </summary>
    /// <example>
    /// Mapping
    ///     TradeStrategy (base) to 
    ///     TradeStrategyNone and TradeStrategyRandom (derived)
    ///     
    /// var converter = new JsonPolymorphicConverter<TradeStrategy>(
    ///     (nameof(TradeStrategyNone), typeof(TradeStrategyNone)),
    ///     (nameof(TradeStrategyRandom), typeof(TradeStrategyRandom)));
    /// var options = new JsonSerializerOptions();
    /// var options.Converters.Add(converter);
    /// </example>
    /// <typeparam name="TItem">Base class type</typeparam>
    public class JsonPolymorphicConverter<TItem> : JsonConverter<TItem>
        where TItem : notnull
    {

        public JsonPolymorphicConverter(params (string key, Type type)[] mappings)
        {
            foreach (var (key, type) in mappings)
                KeyTypeLookup.Add(key, type);
        }

        public ReversibleLookup<string, Type> KeyTypeLookup = new ReversibleLookup<string, Type>();

        public override bool CanConvert(Type typeToConvert)
            => typeof(TItem).IsAssignableFrom(typeToConvert);

        public override TItem Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        {
            // Helper function for validating where you are in the JSON    
            void validateToken(Utf8JsonReader reader, JsonTokenType tokenType)
            {
                if (reader.TokenType != tokenType)
                    throw new JsonException($"Invalid token: Was expecting a '{tokenType}' token but received a '{reader.TokenType}' token");
            }

            TItem result = default(TItem);

            reader.Read(); // Move to property name
            validateToken(reader, JsonTokenType.PropertyName);

            var typeKey = reader.GetString();

            reader.Read(); // Move to start of object (stored in this property)
            validateToken(reader, JsonTokenType.StartObject); // Start of vehicle

            if (KeyTypeLookup.TryGetValue(typeKey, out var concreteItemType))
            {
                // WORKAROUND - stop cyclic look up
                // If we leave our converter in the options then will get infinite cycling
                // We create a temp options with our converter removed to stop the cycle
                JsonSerializerOptions tempOptions = new JsonSerializerOptions(options);
                tempOptions.Converters.Remove(this);

                // Use normal deserialization
                result = (TItem)JsonSerializer.Deserialize(ref reader, concreteItemType, tempOptions);
            }
            else
            {
                throw new JsonException($"Unknown type key '{typeKey}' found");
            }

            reader.Read(); // Move past end of item object

            return result;
        }

        public override void Write(Utf8JsonWriter writer, TItem item, JsonSerializerOptions options)
        {
            var itemType = item.GetType();

            writer.WriteStartObject();

            if (KeyTypeLookup.ReverseLookup.TryGetValue(itemType, out var typeKey))
            {
                writer.WritePropertyName(typeKey);

                // WORKAROUND - stop cyclic look up
                // If we leave our converter in the options then will get infinite cycling
                // We create a temp options with our converter removed to stop the cycle
                JsonSerializerOptions tempOptions = new JsonSerializerOptions(options);
                tempOptions.Converters.Remove(this);

                // Use normal serialization
                JsonSerializer.Serialize(writer, item, itemType, tempOptions);
            }
            else
            {
                throw new JsonException($"Unknown type '{itemType.FullName}' found");
            }

            writer.WriteEndObject();
        }
    }
}

This also relies on the ReversibleLookup class from https://stackoverflow.com/a/59744873. I am copying here for convenience. The code is the same, I've only added a comment at the top.

using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Diagnostics;

namespace Shared.DataAccess
{
    /// <summary>
    /// Helper class used with JsonPolymorphicConverter and HeterogenousListConverter
    /// </summary>
    /// <typeparam name="T1">First class type</typeparam>
    /// <typeparam name="T2">Second class type</typeparam>
    public class ReversibleLookup<T1, T2> : ReadOnlyDictionary<T1, T2>
    where T1 : notnull
    where T2 : notnull
    {

        public ReversibleLookup(params (T1, T2)[] mappings)
        : base(new Dictionary<T1, T2>())
        {

            ReverseLookup = new ReadOnlyDictionary<T2, T1>(reverseLookup);

            foreach (var mapping in mappings)
                Add(mapping.Item1, mapping.Item2);
        }

        private readonly Dictionary<T2, T1> reverseLookup = new Dictionary<T2, T1>();
        public ReadOnlyDictionary<T2, T1> ReverseLookup { get; }

        [DebuggerHidden]
        public void Add(T1 value1, T2 value2)
        {

            if (ContainsKey(value1))
                throw new InvalidOperationException($"{nameof(value1)} is not unique");

            if (ReverseLookup.ContainsKey(value2))
                throw new InvalidOperationException($"{nameof(value2)} is not unique");

            Dictionary.Add(value1, value2);
            reverseLookup.Add(value2, value1);
        }

        public void Clear()
        {
            Dictionary.Clear();
            reverseLookup.Clear();
        }
    }
}

Example usage

public class TradeStrategy { 
    public string name; 
    public TradeStrategy() : this("Unknown") { }
    public TradeStrategy(string name) { this.name = name; }
    public virtual double CalcAdjustments(double stockPrice) => 0.0;
}
public class TradeStrategyNone : TradeStrategy {
    public TradeStrategyNone() : base("None") { }
    public override double CalcAdjustments(double stockPrice) => 0.0;
}
public class TradeStrategyRandom : TradeStrategy {
    private Random random { get; set; }
    public TradeStrategyRandom() : base("Random") { random = new Random(); }
    public override double CalcAdjustments(double stockPrice) => random.NextDouble();
}
public class Portfolio {
    public TradeStrategy strategy;
}

var converter = new JsonPolymorphicConverter<TradeStrategy>(
    (nameof(TradeStrategyNone), typeof(TradeStrategyNone)),
    (nameof(TradeStrategyRandom), typeof(TradeStrategyRandom)));

var options = new JsonSerializerOptions();
options.Converters.Add(converter);

Portfolio port1 = new Portfolio();
port1.strategy = new TradeStrategyRandom();

// port1Json will contain "TradeStrategyRandom" type info for "TradeStrategy" strategy variable
var port1Json = JsonSerializer.Serialize(port1, options);

// port1Copy will properly create "TradeStrategyRandom" from the port1Json
Portfolio port1Copy = JsonSerializer.Deserialize<Portfolio>(port1Json, options);

// Without "options" the JSON will end up stripping down TradeStrategyRandom to TradeStrategy

If you are trying to figure out the difference between this solution and the other one, know that the other solution requires you to create an array of the item you want to convert. This solution will work with a single object.

Rick
  • 21
  • 3
-1

Here's a simplistic approach that I hope works for you.

You can use a dynamic variable

I noticed in the comments that you weren't keen to use NetwonSoft.Json, you could use this code: dynamic car = Json.Decode(json);

The Json class comes from here

Murchiad
  • 103
  • 10
  • Isn't that `Json.NET`? I'm using `System.Text.Json`. – Mark A. Donohoe Jan 15 '20 at 01:58
  • @MarqueIV - It was linked in the documentation. Assembly: System.Web.Helpers.dll – Murchiad Jan 15 '20 at 02:17
  • @MarqueIV Also, it's not the library you use that's important, it's the dynamic object. You could use the System.Text.Json way of serializing/deserializing the object. – Murchiad Jan 15 '20 at 02:25
  • 2
    But the dynamic object is *not* a concrete type! For instance, if you have an enumeration an just use dynamic, you'd end up with a string and not the enumerated value. This is because the JSON has no type information. The solution I propose below returns true, concrete types, albeit via their base class, but you can cast them right back out again, as well as use them as ItemsSources in things like binding and they will match to their concrete types. Dynamic doesn't do any of that. It can't because it doesn't have the needed information. – Mark A. Donohoe Jan 15 '20 at 02:32
  • @MarqueIV - +1 yup that's correct, it is still an option in some scenarios though so thought it'd be worth putting in this thread. – Murchiad Jan 15 '20 at 23:02
  • I can see the benefit in some cases. That said, if you're just blindly reading in JSON into a `dynamic` you may want to look into the new JSON DOM in `System.Text.View` which is still somewhat undefined, but defined enough in the context of JSON model itself. – Mark A. Donohoe Jan 16 '20 at 04:06