use Newtonsoft to parse Configuration files in .net 5

Question

I have a json configuration file where I would like to store a json object that could hold several unknown fields.

I can deserialize such an object using Newtonsoft by deriving from this base class:

public class ComplexJsonObject
{
    [JsonExtensionData]
    private IDictionary<string, JToken> _additionalData_newtonsoft;
}

Unfortunately it seems that the config file appsettings.development.json just says I have an empty object. Even though there is something configured.

I assumed this was because the system used System.Text.Json. So I tried that as well:

public class ComplexJsonObject
{
    [JsonExtensionData]
    private IDictionary<string, JToken> _additionalData_newtonsoft;

    [System.Text.Json.Serialization.JsonExtensionData]
    [Newtonsoft.Json.JsonIgnore]
    public IDictionary<string, JsonElement> _additionalData_dotnet { get; set; }
}

This does not work either.

So the question: how do I tell the system to use Newtonsoft for deserializing this config file?

-- edit --

As requested, an example of the config I would like to store. The config key would be "configuration:when" and the object I expect must have a operator, but all the other fields are dynamic.

{
    "extraction": {
        "when": {
            "operator": "and",
            "rules": [
            {
                "operator": "or",
                "rules": [ 
                { "operator": "field.contains", "value": "waiting" },
                { "operator": "field.contains", "value": "approved" },
                { "operator": "field.contains", "value": "rejected" }
                ]
            },
            { "operator": "not", "rule": { "operator": "field.contains", "value": "imported" } },
            { "operator": "not", "rule": { "operator": "field.contains", "value": "import-failed" } }
            ]
        }
    }
}   

I think Métoule is correct, and this is indeed not possible. Since the config by default would mix values from other files.

Answer 1

If you need to convert just a specific section using newtonsoft, you can use this extension:

public static class ConfigurationBinder
{
    public static void BindJsonNet<T>(this IConfiguration config, T instance) where T : class
    {
        string objectPath = config.AsEnumerable().Select(x => x.Key).FirstOrDefault();

        var obj = BindToExpandoObject(config, objectPath);
        if (obj == null)
            return;

        var jsonText = JsonConvert.SerializeObject(obj);
        var jObj = JObject.Parse(jsonText);
        if (jObj == null)
            return;

        var jToken = jObj.SelectToken($"*.{GetLastObjectName(objectPath)}");
        if (jToken == null)
            return;

        jToken.Populate<T>(instance);
    }

    private static ExpandoObject BindToExpandoObject(IConfiguration config, string objectPath)
    {
        var result = new ExpandoObject();
        string lastObjectPath = GetLastObjectPath(objectPath);

        // retrieve all keys from your settings
        var configs = config.AsEnumerable();
        configs = configs
            .Select(x => new KeyValuePair<string, string>(x.Key.Replace(lastObjectPath, ""), x.Value))
            .ToArray();

        foreach (var kvp in configs)
        {
            var parent = result as IDictionary<string, object>;
            var path = kvp.Key.Split(':');

            // create or retrieve the hierarchy (keep last path item for later)
            var i = 0;
            for (i = 0; i < path.Length - 1; i++)
            {
                if (!parent.ContainsKey(path[i]))
                {
                    parent.Add(path[i], new ExpandoObject());
                }

                parent = parent[path[i]] as IDictionary<string, object>;
            }

            if (kvp.Value == null)
                continue;

            // add the value to the parent
            // note: in case of an array, key will be an integer and will be dealt with later
            var key = path[i];
            parent.Add(key, kvp.Value);
        }

        // at this stage, all arrays are seen as dictionaries with integer keys
        ReplaceWithArray(null, null, result);

        return result;
    }

    private static string GetLastObjectPath(string objectPath)
    {
        string lastObjectPath = objectPath;
        int indexLastObj;
        if ((indexLastObj = objectPath.LastIndexOf(":")) != -1)
            lastObjectPath = objectPath.Remove(indexLastObj);
        return lastObjectPath;
    }

    private static string GetLastObjectName(string objectPath)
    {
        string lastObjectPath = objectPath;
        int indexLastObj;
        if ((indexLastObj = objectPath.LastIndexOf(":")) != -1)
            lastObjectPath = objectPath.Substring(indexLastObj + 1);
        return lastObjectPath;
    }

    private static void ReplaceWithArray(ExpandoObject parent, string key, ExpandoObject input)
    {
        if (input == null)
            return;

        var dict = input as IDictionary<string, object>;
        var keys = dict.Keys.ToArray();

        // it's an array if all keys are integers
        if (keys.All(k => int.TryParse(k, out var dummy)))
        {
            var array = new object[keys.Length];
            foreach (var kvp in dict)
            {
                array[int.Parse(kvp.Key)] = kvp.Value;
            }

            var parentDict = parent as IDictionary<string, object>;
            parentDict.Remove(key);
            parentDict.Add(key, array);
        }
        else
        {
            foreach (var childKey in dict.Keys.ToList())
            {
                ReplaceWithArray(input, childKey, dict[childKey] as ExpandoObject);
            }
        }
    }

    public static void Populate<T>(this JToken value, T target) where T : class
    {
        using (var sr = value.CreateReader())
        {
            JsonSerializer.CreateDefault().Populate(sr, target); // Uses the system default JsonSerializerSettings
        }
    }
}

Usage:

var obj = new SampleObject();
Configuration.GetSection("test:arr:3:sampleObj").BindJsonNet(obj);
services.AddSingleton(obj);

Reference:

https://stackoverflow.com/a/55050425/7064571

Bind netcore IConfigurationSection to a dynamic object

Answer 2

What you want is not possible, because that's not how the .NET configuration system works. The configuration doesn't directly parse your JSON into your data structure; instead, it creates an intermediate representation (which is a simple IDictionary<string,string>) which is then bound to your data structure.

The reason why it's not a direct mapping is because the configuration data can come from multiple sources. For example, it's possible to override the JSON configuration with values specified via the Azure portal UI. Or there might not be a JSON file at all.

That being said, it's possible to abuse the configuration system, like I explained in the following questions:

• How to load polymorphic objects in appsettings.json
• Bind netcore IConfigurationSection to a dynamic object