Deserialize nested JObject with the key into List

Question

I need to deserialize a json back to object instance using Newtonsoft.Json.

However, it is a list type object and the key of the entry is useful to me.

I don't know how to deserialize automatically without mapping the fields one by one manually.

Here is the response:

{
    coins: {
        365Coin: {
            id: 74,
            tag: "365",
            algorithm: "Keccak",
            lagging: true,
            listed: false,
            status: "No available stats",
            testing: false
        },
        Aiden: {
            id: 65,
            tag: "ADN",
            algorithm: "Scrypt-OG",
            lagging: true,
            listed: false,
            status: "No available stats",
            testing: false
        },
        Adzcoin: {
            id: 157,
            tag: "ADZ",
            algorithm: "X11",
            lagging: false,
            listed: false,
            status: "Active",
            testing: false
        }
        ... [With various key representing the name of coins]
    }
}

The full response: https://whattomine.com/calculators.json

My best guess of the class is somethings like:

internal class WhatToMineCalculatorsResponse
{
    // Should be Dictionary???
    [JsonProperty("coins")]
    public IList<WhatToMineCalculatorResponse> Coins { get; set; }
}

internal class WhatToMineCalculatorResponse
{
    // I want the key set in this field
    public string Name { get; set; }

    [JsonProperty("id")]
    public int Id { get; set; }

    [JsonProperty("tag")]
    public string Symbol { get; set; }

    [JsonProperty("status")]
    public string Status { get; set; }

    [JsonProperty("algorithm")]
    public string Algo { get; set; }

    [JsonProperty("listed")]
    public bool IsListed { get; set; }
}

Note that I want the key included in my class but not as the key of a dictionary. It's hard to retrieve the key later.

Answer 1

You cannot specify entirely through attributes that an IList<T> for some T should be serialized as a JSON object. As explained in its Serialization Guide, Newtonsoft maps dictionaries and hashtables to JSON objects, but maps all other enumerables, lists and arrays to JSON arrays. Instead, you will have to use a custom JsonConverter.

First, define the following converter:

internal class WhatToMineCalculatorResponseListConverter : KeyedListToJsonObjectConverterBase<WhatToMineCalculatorResponse>
{
    protected override string KeyPropertyUnderlyingName => nameof(WhatToMineCalculatorResponse.Name);
}

public abstract class KeyedListToJsonObjectConverterBase<T> : JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        if (objectType.IsArray)
            return false;
        return typeof(IList<T>).IsAssignableFrom(objectType);
    }

    protected abstract string KeyPropertyUnderlyingName { get; }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        // Get the key property name from the underlying name
        var itemContract = serializer.ContractResolver.ResolveContract(typeof(T)) as JsonObjectContract;
        if (itemContract == null)
            throw new JsonSerializationException(string.Format("type {0} is not serialized as a JSON object"));
        var keyProperty = itemContract.Properties.Where(p => p.UnderlyingName == KeyPropertyUnderlyingName).SingleOrDefault();
        if (keyProperty == null)
            throw new JsonSerializationException(string.Format("Key property {0} not found", KeyPropertyUnderlyingName));

        // Validate initial token.
        if (reader.SkipComments().TokenType == JsonToken.Null)
            return null;
        if (reader.TokenType != JsonToken.StartObject)
            throw new JsonSerializationException(string.Format("Unexpected token {0} at {1}", reader.TokenType, reader.Path));

        // Allocate the List<T>.  (It might be some subclass of List<T>, so use the default creator.
        var list = existingValue as ICollection<T> ?? (ICollection<T>)serializer.ContractResolver.ResolveContract(objectType).DefaultCreator();

        // Process each key/value pair.
        while (reader.Read())
        {
            switch (reader.TokenType)
            {
                case JsonToken.Comment:
                    break;
                case JsonToken.EndObject:
                    return list;
                case JsonToken.PropertyName:
                    {
                        // Get the name.
                        var name = (string)reader.Value;
                        reader.ReadAndAssert();
                        // Load the object
                        var jItem = JObject.Load(reader);
                        // Add the name property
                        jItem.Add(keyProperty.PropertyName, name);
                        // Deserialize the item and add it to the list.
                        list.Add(jItem.ToObject<T>(serializer));
                    }
                    break;
                default:
                    {
                        throw new JsonSerializationException(string.Format("Unexpected token {0} at {1}", reader.TokenType, reader.Path));
                    }
            }
        }
        // Should not come here.
        throw new JsonSerializationException("Unclosed object at path: " + reader.Path);
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        // Get the key property name from the underlying name
        var itemContract = serializer.ContractResolver.ResolveContract(typeof(T)) as JsonObjectContract;
        if (itemContract == null)
            throw new JsonSerializationException(string.Format("type {0} is not serialized as a JSON object"));
        var keyProperty = itemContract.Properties.Where(p => p.UnderlyingName == KeyPropertyUnderlyingName).SingleOrDefault();
        if (keyProperty == null)
            throw new JsonSerializationException(string.Format("Key property {0} not found", KeyPropertyUnderlyingName));

        var converters = serializer.Converters.ToArray();
        var list = (IEnumerable<T>)value;
        writer.WriteStartObject();
        foreach (var item in list)
        {
            var jItem = JObject.FromObject(item, serializer);
            var name = (string)jItem[keyProperty.PropertyName];
            jItem.Remove(keyProperty.PropertyName);
            writer.WritePropertyName(name);
            jItem.WriteTo(writer, converters);
        }
        writer.WriteEndObject();
    }
}

public static partial class JsonExtensions
{
    public static JsonReader SkipComments(this JsonReader reader)
    {
        while (reader.TokenType == JsonToken.Comment && reader.Read())
            ;
        return reader;
    }

    public static void ReadAndAssert(this JsonReader reader)
    {
        if (reader == null)
            throw new ArgumentNullException();
        if (!reader.Read())
        {
            new JsonReaderException(string.Format("Unexpected end at path {0}", reader.Path));
        }
    }
}

Then, you can deserialize as follows:

var settings = new JsonSerializerSettings
{
    Converters = { new WhatToMineCalculatorResponseListConverter() },
};
var root = JsonConvert.DeserializeObject<WhatToMineCalculatorsResponse>(responseString, settings);

Notes:

• The base class converter KeyedListToJsonObjectConverterBase<T> can be reused in any case where where you are serializing a List<T> and the type T has a specific property to be used as a JSON object property name. Simply override KeyPropertyUnderlyingName and return the actual .Net property name (not the serialized name).

• The code looks a bit complicated because I made KeyedListToJsonObjectConverterBase<T> general enough to handle situations in which the key property is read-only, such as:

  internal class WhatToMineCalculatorResponse
  {
      readonly string _name;
  
      public WhatToMineCalculatorResponse(string name)
      {
          this._name = name;
      }
  
      // I want the key set in this field
      public string Name { get { return _name; } }
  
      // Remainder of class unchanged
  }
  

Working .Net fiddle here.