How to deserialize JSON array with changeable number of subarrays using System.Text.Json in C#

Question

Is it possible to automatically deserialize JSON to an array of arrays even if that JSON sometimes only contains one array element and thus doesn't technically represent an array of arrays but a single array element?

The json is a chart that should have a number of [x, y] point values but every once in a while it only has one point.

Example data when it only contains one array element:

{"data" : { "chart": [ 1, 1 ] } }

and when it contains an array of arrays it looks usually something like this:

{"data": {"chart": [ [ 1, 1 ],  [ 2, 2 ], [ 3, 3 ], [ 4, 4 ]]}}

I realize I can deserialize it into this:

public class Data
{
    public List<object> chart { get; set; }
}

public class Root
{
    public Data data { get; set; }
}

but I'd rather like to have a List<List<double>> even if it contained just one array element with the outer List, obviously containing just the List/array with the one element/point.

• Is that doable by some custom converter through System.Text.Json and how please?
• Also is it doable for multiple charts data in JSON i.e. if I have "chart1", "chart2", "chart3" etc.

TIA

P.S. I don't have access to code of the JSON producer and unfortunately it only works like described when there's just one point (so no I can't have it return [[x,y]] instead of [x,y] if that's even valid JSON.

Answer 1

Assuming your JSON array is only ever a 2D or 1D array, you may declare your c# property as a jagged 2D list List<List<T>>, and apply a custom JsonConverter that checks to see whether the incoming JSON is a 1D or 2D array, and if not 2D, deserializes a single List<T> and returns a List<List<T>> containing that single list.

First, introduce the following converter factory and converter:

public class Jagged2DOr1DListConverter : JsonConverterFactory
{
    public virtual bool CanWrite => true;
    
    public override bool CanConvert(Type typeToConvert) => Get2DListItemType(typeToConvert) != null;

    public override JsonConverter CreateConverter(Type typeToConvert, JsonSerializerOptions options)
    {
        var itemType = Get2DListItemType(typeToConvert)!;
        var converterType = typeof(Jagged2DOr1DListConverter<>).MakeGenericType(itemType);
        return (JsonConverter)Activator.CreateInstance(converterType, new object [] { CanWrite })!;
    }
    
    static Type? Get2DListItemType(Type type)
    {
        if (!type.IsGenericType || type.GetGenericTypeDefinition() != typeof(List<>))
            return null;
        var midType = type.GetGenericArguments()[0];
        if (!midType.IsGenericType || midType.GetGenericTypeDefinition() != typeof(List<>))
            return null;
        var itemType = midType.GetGenericArguments()[0];
        if (itemType == typeof(string) || itemType == typeof(byte []))
            return itemType;
        // values declared as object are serialized polymorphically, so we can't say whether they will appear as JSON arrays, objects or primitives.  
        // So don't convert List<List<object>> automatically.
        if (itemType == typeof(object))
            return null; 
        // The following check could be enhanced to detect and allow conversion of List<List<Dictionary<TKey, TValue>>> types, since these are serialized as JSON objects rather than arrays.
        if (typeof(IEnumerable).IsAssignableFrom(itemType))
            return null;
        return itemType;
    }
}

public class Jagged2DOr1DListReadOnlyConverter : Jagged2DOr1DListConverter
{
    public override bool CanWrite => false;
}

public class Jagged2DOr1DListConverter<TItem> : JsonConverter<List<List<TItem>>>
{
    public Jagged2DOr1DListConverter() : this(true) {}
    public Jagged2DOr1DListConverter(bool canWrite) => CanWrite = canWrite;

    public bool CanWrite { get; }

    public override List<List<TItem>>? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        if (reader.TokenType == JsonTokenType.Null)
            return null;
        else if (reader.TokenType != JsonTokenType.StartArray)
            throw new JsonException();
        List<List<TItem>> outerList = new();
        List<TItem>? innerList = null;

        while (reader.Read() && reader.TokenType != JsonTokenType.EndArray)
        {
            if (reader.TokenType == JsonTokenType.StartArray)
            {
                if (outerList.Count > 0 && innerList != null)
                    throw new JsonException();  // Mixtures of items and lists of items are not implemented
                var itemList = JsonSerializer.Deserialize<List<TItem>>(ref reader, options); // You could throw an exception here if itemList is null, if you want
                outerList.Add(itemList!); // You could throw an exception here if the 
            }
            else
            {
                if (innerList == null)
                {
                    if (outerList.Count > 0)
                        throw new JsonException();  // Mixtures of items and lists of items are not implemented
                    outerList.Add(innerList = new ());
                }
                innerList.Add(JsonSerializer.Deserialize<TItem>(ref reader, options)!);
            }
        }
        return outerList;
    }

    public override void Write(Utf8JsonWriter writer, List<List<TItem>> value, JsonSerializerOptions options)
    {
        if (CanWrite && value.Count == 1)
        {
            JsonSerializer.Serialize(writer, value.First(), options);
        }
        else
        {
            writer.WriteStartArray();
            foreach (var item in value)
                JsonSerializer.Serialize(writer, item, options);
            writer.WriteEndArray();
        }
    }
}

Then, modify Data as follows:

public class Data
{
    public List<List<decimal>> chart { get; set; } = new (); // Use int, double, float or decimal here, as required.
}

And now you will be able to deserialize by adding Jagged2DOr1DListConverter to JsonSerializerOptions as follows:

var options = new JsonSerializerOptions
{
    Converters = { new Jagged2DOr1DListConverter() },
    // Other    
    WriteIndented = true,
};
var root = JsonSerializer.Deserialize<Root>(json, options);

Alternatively, you could apply the converter directly to Data.chart like so:

public class Data
{
    [JsonConverter(typeof(Jagged2DOr1DListConverter))]
    public List<List<decimal>> chart { get; set; } = new (); // Use int, double, float or decimal here, as required.
}

Notes:

• If you don't want 2D lists with a single item to be re-serialized as 1D JSON arrays, use Jagged2DOr1DListReadOnlyConverter.

• Jagged2DOr1DListConverter will automatically convert all List<List<T>> types, where T is not itself an enumerable. If you only want specific types to be converted, add instance(s) of Jagged2DOr1DListConverter<TItem> to JsonSerializerOptions.Converters instead.

• If you want to disallow null values for the outer list, override Jagged2DOr1DListConverter<TItem>.HandleNull and throw an exception from Read() when reader.TokenType == JsonTokenType.Null.

• Lists of lists of dictionaries won't be automatically converted by Jagged2DOr1DListConverter. If necessary, you could enhance this check in Get2DListItemType(Type type) to return a non-null item type for dictionary items:

   // The following check could be enhanced to detect and allow conversion of List<List<Dictionary<TKey, TValue>>>` types, since these are serialized as JSON objects rather than arrays.
   if (typeof(IEnumerable).IsAssignableFrom(itemType))
       return null;
  

• List<List<object>> is also not converted automatically. System.Text.Json serialized values declared as object polymorphically, and so inner items might themselves be serialized as arrays.

Demo fiddle here.