Below is a solution I came up with that works pretty well.
Had some sample JSON similar to below. JSON was very flat and you can see the indexed Check/Transfer properties. I wanted to allow NewtonSoft to do as much of the heavy lifting as possible.
{
"id": "209348",
"Check__00__amount": 10000,
"Check__00__payableTo": "ABC Company",
"Check__00__receivedFrom": "Mike",
"Check__01__amount": 20000,
"Check__01__payableTo": "XYZ Company",
"Check__01__receivedFrom": "Jim",
"Transfer00__Amount": 50000.0,
"Transfer00__CompanyTransferringFrom": "DEF Company",
"Transfer00__Type": "Partial",
"Transfer01__Amount": 55000.0,
"Transfer01__CompanyTransferringFrom": "GHI Company",
"Transfer01__Type": "Full"
}
Types to deserialize to. Top level object Transaction with two list properties for the checks and transfers. I'm using a custom converter and a custom attribute.
[JsonConverter(typeof(TestConverter))]
public class Transaction
{
[JsonProperty("id")]
public int Id { get; set; } = default!;
[RegexIndexedPropertiesToList(@"Transfer\d+__")]
public List<Transfer> Transfers { get; set; } = default!;
[RegexIndexedPropertiesToList(@"Check__\d+__")]
public List<Check> Checks { get; set; } = default!;
}
public class Check
{
[JsonProperty("amount")]
public decimal Amount { get; set; }
[JsonProperty("payableTo")]
public string PayableTo { get; set; } = default!;
[JsonProperty("receivedFrom")]
public string ReceivedFrom { get; set; } = default!;
}
public class Transfer
{
[JsonProperty("Amount")]
public decimal Amount { get; set; }
[JsonProperty("CompanyTransferringFrom")]
public string CompanyTransferringFrom { get; set; } = default!;
[JsonProperty("Type")]
public string Type { get; set; } = default!;
}
custom attribute. Allows for the setting of the regex. This regex should be able to Match the properties that you want to turn into a list. You can see it decorated on the Checks/List properties in the Transaction class.
[AttributeUsage(AttributeTargets.Property, AllowMultiple = false)]
public class RegexIndexedPropertiesToListAttribute : Attribute
{
public string Regex { get; }
public RegexIndexedPropertiesToListAttribute(string regex)
{
Regex = regex;
}
}
The converter
public class TestConverter : JsonConverter
{
public override void WriteJson(JsonWriter writer, object? value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
public override object? ReadJson(JsonReader reader, Type objectType, object? existingValue, JsonSerializer serializer)
{
var obj = JObject.Load(reader);
//get a new instance and populate it to prevent infinite recursion on the converter, this will allow default serializer behavior to work
var returnable = Activator.CreateInstance(objectType) ?? throw new InvalidOperationException("could not create instance");
JsonConvert.PopulateObject(obj.ToString(), returnable); //use regular deserialization
IndexedPropertiesToList(returnable, obj);
return returnable;
}
private static void IndexedPropertiesToList(object returnable, JObject obj)
{
//get all the index to list properties
var propsForConversion = returnable.GetType().GetProperties()
.Where(x => x.IsDefined(typeof(RegexIndexedPropertiesToListAttribute), false));
foreach (var prop in propsForConversion)
{
var attribute = (prop.GetCustomAttributes(typeof(RegexIndexedPropertiesToListAttribute), false).FirstOrDefault() as RegexIndexedPropertiesToListAttribute)
?? throw new Exception("attribute not found");
//assume the prperty to be set is a list
var list = Activator.CreateInstance(prop.PropertyType) as IList ??
throw new InvalidOperationException("could not create instance");
var regex = new Regex(attribute.Regex);
//get the properties that match the regex and then group them using the regex as a prefix
var matchedProperties = obj.Properties().Where(x => regex.IsMatch(x.Path)).ToList();
var groups = matchedProperties.GroupBy(x => regex.Match(x.Path).Value).ToList();
foreach (var group in groups)
{
var newObj = new JObject(); //create a new jobject will use this to deserialize the properties into type so that normal deserialization works
foreach (var property in group)
{
var name = property.Name.Replace(group.Key, "");
newObj.Add(name, property.Value); //add the property to the new object with no index
}
//assumes the List is of a generic type
var genericType = prop.PropertyType.GenericTypeArguments[0];
var instance = newObj.ToObject(genericType) ??
throw new InvalidOperationException("could not deserialize");
list.Add(instance);
}
//set the constructed list of deserialized objects to the property
prop.SetValue(returnable, list);
}
}
public override bool CanConvert(Type objectType)
{
throw new NotImplementedException();
}
}
First the converter populates the top level in this case the Transaction. Then it looks for properties that are decorated with the attribute. It uses that regex to match fields from the JSON and then groups by the match portion, Check__00__ and Check__01__ would be two groupings with 3 JTokens each for the corresponding properties. From there it builds a new JObject with the grouped fields. The JObject will have the regex matched portion of the name removed which allows for the usage of JsonProperty attributes for mapping. Then it uses the default Newtonsoft deserialization to get a new instance of the List type and adds it to the list, then sets the List property on the top level instance.
Its worked well for what I've needed it for.
The good
Uses regular NewtonSoft deserialzation
only requires the custom converter and the attribute
Seems to execute pretty quickly on some sizable payloads.
What I have above assumes it is deeling with a List and further adaptation would be needed if that weren't the case. It wouldn't be able to handle deeper nesting, though I think it could be added if needed, and probably wouldn't play well with List of primitives, though I think that could be handled as well with a bit of additional code.
