C# flattening json structure

Question

I have a json-object in C# (represented as a Newtonsoft.Json.Linq.JObject object) and I need to flatten it to a dictionary. Let me show you an example of what I mean:

{
    "name": "test",
    "father": {
         "name": "test2"
         "age": 13,
         "dog": {
             "color": "brown"
         }
    }
}

This should yield a dictionary with the following key-value-pairs:

["name"] == "test",
["father.name"] == "test2",
["father.age"] == 13,
["father.dog.color"] == "brown"

How can I do this?

Answer 1

JObject jsonObject=JObject.Parse(theJsonString);
IEnumerable<JToken> jTokens = jsonObject.Descendants().Where(p => !p.HasValues);
Dictionary<string, string> results = jTokens.Aggregate(new Dictionary<string, string>(), (properties, jToken) =>
                    {
                        properties.Add(jToken.Path, jToken.ToString());
                        return properties;
                    });

I had the same requirement of flattening a nested json structure to a dictionary object. Found the solution here.

Answer 2

As of .NET Core 3.0 JsonDocument is a way (Json.NET is not needed). I'm sure this will get easier.

using System.Linq;
using System.Text.Json;
(...)


public static Dictionary<string, JsonElement> GetFlat(string json)
{
    IEnumerable<(string Path, JsonProperty P)> GetLeaves(string path, JsonProperty p)
        => p.Value.ValueKind != JsonValueKind.Object
            ? new[] { (Path: path == null ? p.Name : path + "." + p.Name, p) }
            : p.Value.EnumerateObject() .SelectMany(child => GetLeaves(path == null ? p.Name : path + "." + p.Name, child));

    using (JsonDocument document = JsonDocument.Parse(json)) // Optional JsonDocumentOptions options
        return document.RootElement.EnumerateObject()
            .SelectMany(p => GetLeaves(null, p))
            .ToDictionary(k => k.Path, v => v.P.Value.Clone()); //Clone so that we can use the values outside of using
}

A more expressive version is shown below.

Test

using System.Linq;
using System.Text.Json;
(...)

var json = @"{
    ""name"": ""test"",
    ""father"": {
            ""name"": ""test2"", 
         ""age"": 13,
         ""dog"": {
                ""color"": ""brown""
         }
        }
    }";

var d = GetFlat(json);
var options2 = new JsonSerializerOptions { WriteIndented = true };
Console.WriteLine(JsonSerializer.Serialize(d, options2));

Output

{
  "name": "test",
  "father.name": "test2",
  "father.age": 13,
  "father.dog.color": "brown"
}

More expressive version

using System.Linq;
using System.Text.Json;
(...)

static Dictionary<string, JsonElement> GetFlat(string json)
    {
        using (JsonDocument document = JsonDocument.Parse(json))
        {
            return document.RootElement.EnumerateObject()
                .SelectMany(p => GetLeaves(null, p))
                .ToDictionary(k => k.Path, v => v.P.Value.Clone()); //Clone so that we can use the values outside of using
        }
    }


    static IEnumerable<(string Path, JsonProperty P)> GetLeaves(string path, JsonProperty p)
    {
        path = (path == null) ? p.Name : path + "." + p.Name;
        if (p.Value.ValueKind != JsonValueKind.Object)
            yield return (Path: path, P: p);
        else
            foreach (JsonProperty child in p.Value.EnumerateObject())
                foreach (var leaf in GetLeaves(path, child))
                    yield return leaf;
    }

Answer 3

I actually had the same problem earlier today couldn't find this question on SO at first, and ended up writing my own extension method to return the JValue objects containing the leaf node values of the JSON blob. It's similar to the accepted answer, except for some improvements:

1. It handles any JSON you give it (arrays, properties, etc) instead of just a JSON object.
2. Less memory usage
3. No calls to .Count() on descendants you ultimately don't need

Depending on your use case, those may or may not be relevant, but they are for my case. I wrote about learning to flatten the JSON.NET objects on my blog. Here is the extension method I wrote:

public static class JExtensions
{
    public static IEnumerable<JValue> GetLeafValues(this JToken jToken)
    {
        if (jToken is JValue jValue)
        {
            yield return jValue;
        }
        else if (jToken is JArray jArray)
        {
            foreach (var result in GetLeafValuesFromJArray(jArray))
            {
                yield return result;
            }
        }
        else if (jToken is JProperty jProperty)
        {
            foreach (var result in GetLeafValuesFromJProperty(jProperty))
            {
                yield return result;
            }
        }
        else if (jToken is JObject jObject)
        {
            foreach (var result in GetLeafValuesFromJObject(jObject))
            {
                yield return result;
            }
        }
    }

    #region Private helpers

    static IEnumerable<JValue> GetLeafValuesFromJArray(JArray jArray)
    {
        for (var i = 0; i < jArray.Count; i++)
        {
            foreach (var result in GetLeafValues(jArray[i]))
            {
                yield return result;
            }
        }
    }

    static IEnumerable<JValue> GetLeafValuesFromJProperty(JProperty jProperty)
    {
        foreach (var result in GetLeafValues(jProperty.Value))
        {
            yield return result;
        }
    }

    static IEnumerable<JValue> GetLeafValuesFromJObject(JObject jObject)
    {
        foreach (var jToken in jObject.Children())
        {
            foreach (var result in GetLeafValues(jToken))
            {
                yield return result;
            }
        }
    }

    #endregion
}

Then in my calling code, I just extract the Path and Value properties from the JValue objects returned:

var jToken = JToken.Parse("blah blah json here"); 
foreach (var jValue in jToken.GetLeafValues()) 
{
    Console.WriteLine("{0} = {1}", jValue.Path, jValue.Value);
}

Answer 4

You can use https://github.com/jsonfx/jsonfx to deserialize json into a dynamic object. Then use the ExpandoObject to get what you want.

public Class1()
        {
            string json = @"{
                                ""name"": ""test"",
                                ""father"": {
                                     ""name"": ""test2"",
                                     ""age"": 13,
                                     ""dog"": {
                                         ""color"": ""brown""
                                     }
                                }
                            }";

            var reader = new JsonFx.Json.JsonReader();
            dynamic output = reader.Read(json);
            Dictionary<string, object> dict = new Dictionary<string, object>();

            GenerateDictionary((System.Dynamic.ExpandoObject) output, dict, "");
        }

        private void GenerateDictionary(System.Dynamic.ExpandoObject output, Dictionary<string, object> dict, string parent)
        {
            foreach (var v in output)
            {
                string key = parent + v.Key;
                object o = v.Value;

                if (o.GetType() == typeof(System.Dynamic.ExpandoObject))
                {
                    GenerateDictionary((System.Dynamic.ExpandoObject)o, dict, key + ".");
                }
                else
                {
                    if (!dict.ContainsKey(key))
                    {
                        dict.Add(key, o);
                    }
                }
            }
        }

Answer 5

You could use the JSONPath $..* to get all members of the JSON structure and filter out the ones with no children to skip the container properties.

e.g.

var schemaObject = JObject.Parse(schema);
var values = schemaObject
    .SelectTokens("$..*")
    .Where(t => !t.HasValues)
    .ToDictionary(t => t.Path, t => t.ToString());

Answer 6

Based on the code that was provided by tymtam, but also supporting arrays:

    public static IEnumerable<KeyValuePair<string, string>> Flatten<T>(this T data, string seperator = ":") where T : class
    {
        var json = JsonSerializer.Serialize(data);

        string GetArrayPath(string path, string name, int idx) =>
            path == null ? $"{name}{seperator}{idx}" : $"{path}{seperator}{name}{seperator}{idx}";
        IEnumerable<(string Path, JsonElement Element)> GetLeaves(string path, string name, JsonElement element) => element.ValueKind switch
        {
            JsonValueKind.Object => element.EnumerateObject()
                .SelectMany(child => GetLeaves(path == null ? name : $"{path}{seperator}{name}", child.Name, child.Value)),
            JsonValueKind.Array => element.EnumerateArray()
                .SelectMany((child, idx) => child.ValueKind == JsonValueKind.Object
                        ? child.EnumerateObject().SelectMany(child => GetLeaves(GetArrayPath(path, name, idx), child.Name, child.Value))
                        : new[] { (Path: GetArrayPath(path, name, idx), child) }
                    ),
            _ => new[] { (Path: path == null ? name : $"{path}{seperator}{name}", element) },
        };

        using JsonDocument document = JsonDocument.Parse(json); // Optional JsonDocumentOptions options
        return document.RootElement.EnumerateObject()
            .SelectMany(p => GetLeaves(null, p.Name, p.Value))
            .ToDictionary(k => k.Path, v => v.Element.Clone().ToString()); //Clone so that we can use the values outside of using
    }