Combine two JSON structures into one in C#

Question

We have two similar but slightly different JSON structures. First JSON structure is for No Strata scenario – this is applicable for measureId “001” and “002” . Second JSON structure is for multi strata – this is applicable for "measureId": "003". For each testTIN, we need to combine these two measurements as shown in the expected JSON structure.

JSON 1 – No Strata

{
    "testTIN": "123",
    "measurements": [{
            "measureId": "001",
            "value": {
                "IsEndToEndReported": true,
                "PerformanceMet": 5
            }
        },
        {
            "measureId": "002",
            "value": {
                "IsEndToEndReported": true,
                "PerformanceMet": 6
            }

        }
    ]
}

JSON 2 – Multi Strata

{
    "testTIN": "123",
    "measurements": [
       {
        "measureId": "003",
        "value": {
            "strata": [{
                    "IsEndToEndReported": true,
                    "PerformanceMet": 5,
                    "Stratum": "Level1"
                },
                {
                    "IsEndToEndReported": true,
                    "PerformanceMet": 6,
                    "Stratum": "Level2"
                }
            ]
        }
    }
   ]
}

Expected JSON

{
    "testTIN": "123",
    "measurements": [{
            "measureId": "001",
            "value": {
                "IsEndToEndReported": true,
                "PerformanceMet": 5
            }
        },
        {
            "measureId": "002",
            "value": {
                "IsEndToEndReported": true,
                "PerformanceMet": 6
            }
        },
        {
            "measureId": "003",
            "value": {
                "strata": [{
                        "IsEndToEndReported": true,
                        "PerformanceMet": 5,
                        "Stratum": "Level1"
                    },
                    {
                        "IsEndToEndReported": true,
                        "PerformanceMet": 6,
                        "Stratum": "Level2"
                    }
                ]
            }
        }
    ]
}

How to get this new JSON structure combining both the above JSON structures?

C# Code

//No Strata
        List<FlattenedRawData> rowList_NoStrata = HelperMethodClasses.GetFlattenedRawData_NoStrata();
        List<MeasurementSet__NoStrata> result_NoStrata = rowList_NoStrata.GroupBy(records => records.EffectiveTIN)
                     .Select(y => new MeasurementSet__NoStrata
                     {
                         testTIN = y.Key,
                         measurements = y.Select(i =>
                                 new Measurement_NoStrata()
                                 {
                                     measureId = i.MeasureID,
                                     value = new QualityMeasureValue_NoStrata
                                     {
                                         IsEndToEndReported = true,
                                         PerformanceMet = i.PerformanceMetCount
                                     }
                                 })
                            .ToList()
                     })
                .ToList();

        //Multi Strata
        List<FlattenedRawData> rowList_MultiStrata = HelperMethodClasses.GetFlattenedRawData_MultiStrata();
        List<MeasurementSet__MultiStrata> resul_MultiStrata =
                        rowList_MultiStrata.GroupBy(groupBy1 => groupBy1.EffectiveTIN)
                        .Select(level1 => new MeasurementSet__MultiStrata
                        {
                            testTIN = level1.Key,
                            measurements = level1.GroupBy(groupBy2 => groupBy2.MeasureID).Select(level2 =>
                            new Measurement_MultiStrata()
                            {
                                measureId = level2.Key,
                                value = new QualityMeasureValue_MultiStrata()
                                {
                                    strata = level2.Select(level3 => new Strata
                                    {
                                        IsEndToEndReported = true,
                                        PerformanceMet = level3.PerformanceMetCount,
                                        Stratum = level3.Stratum 
                                    }).ToList(),
                                }
                            }).ToList()
                        }).ToList();


        string requestJson = Newtonsoft.Json.JsonConvert.SerializeObject(resul_MultiStrata[0]);

public class FlattenedRawData
{
    public string EffectiveTIN { get; set; }
    public string MeasureID { get; set; }
    public int PerformanceMetCount { get; set; }
    public string Stratum { get; set; }
}

public class Measurement_NoStrata
{
    public string measureId { get; set; }
    public QualityMeasureValue_NoStrata value { get; set; }
}

public class Measurement_MultiStrata
{
    public string measureId { get; set; }
    public QualityMeasureValue_MultiStrata value { get; set; }
}


public class QualityMeasureValue_NoStrata
{
    public bool IsEndToEndReported { get; set; }
    public int PerformanceMet { get; set; }
}


public class QualityMeasureValue_MultiStrata
{
    public List<Strata> strata = new List<Strata>();
}

public class Strata
{
    public bool IsEndToEndReported { get; set; }
    public int PerformanceMet { get; set; }
    public string Stratum { get; set; }
}


public class MeasurementSet__NoStrata
{
    public string testTIN { get; set; }
    public List<Measurement_NoStrata> measurements { get; set; }
}

public class MeasurementSet__MultiStrata
{
    public string category { get; set; }
    public string testTIN { get; set; }
    public List<Measurement_MultiStrata> measurements { get; set; }
}

Update

Good References:

Newtonsoft Json.Net serialize JObject doesn't ignore nulls, even with the right settings

I'd say, you don't need to de-serialize the input JSONs, but parse them. So: 1. Get JToken by Parse method for both JSONs. 2. Get measurements collection sub-token from second JToken collection. 3. Insert it into the first measurements collection. 4. Deserialize the whole thing. — Max, Jan 07 '19 at 02:32
*We have two similar but slightly different JSON structures.* -- do you really have your initial inputs as JSON, or as c# POCOs? — dbc, Jan 07 '19 at 03:34
1) Well then is this really a JSON question, or more of a data modelling question? 2) I notice `MeasurementSet__MultiStrata` has a property `category` not present in `MeasurementSet__NoStrata`. How does this affect merging? — dbc, Jan 07 '19 at 05:05

Gauravsa · Answer 1 · 2019-01-07T03:51:36.090

You can simply do a union (without creating complex POCO classes unless you really need it). Newtonsoft supports merging JSon's:

var dataObject1 = JObject.Parse(@"{
            ""testTIN"" : ""123"",
            ""measurements"": [{
                ""measureId"": ""001"",
                ""value"": {
                    ""IsEndToEndReported"": true,
                    ""PerformanceMet"": 5
                }
            },
            {
                ""measureId"": ""002"",
                ""value"": {
                    ""IsEndToEndReported"": true,
                    ""PerformanceMet"": 6
                }
            }
            ]
        }");

        var dataObject2 = JObject.Parse(@"{
        ""testTIN"": ""123"",
        ""measurements"": [
        {
            ""measureId"": ""003"",
            ""value"": {
                ""strata"": [{
                    ""IsEndToEndReported"": true,
                    ""PerformanceMet"": 5,
                    ""Stratum"": ""Level1""
                },
                {
                    ""IsEndToEndReported"": true,
                    ""PerformanceMet"": 6,
                    ""Stratum"": ""Level2""
                }
                ]
            }
        }
        ]
        }");

        dataObject1.Merge(dataObject2, new JsonMergeSettings
        {
            // union array values together to avoid duplicates
            MergeArrayHandling = MergeArrayHandling.Union
        });

        string json = dataObject1.ToString();

This will give an output:

    {
  "testTIN": "123",
  "measurements": [
    {
      "measureId": "001",
      "value": {
        "IsEndToEndReported": true,
        "PerformanceMet": 5
      }
    },
    {
      "measureId": "002",
      "value": {
        "IsEndToEndReported": true,
        "PerformanceMet": 6
      }
    },
    {
      "measureId": "003",
      "value": {
        "strata": [
          {
            "IsEndToEndReported": true,
            "PerformanceMet": 5,
            "Stratum": "Level1"
          },
          {
            "IsEndToEndReported": true,
            "PerformanceMet": 6,
            "Stratum": "Level2"
          }
        ]
      }
    }
  ]
}

dbc · Accepted Answer · 2019-01-08T23:04:11.920

If your initial No Strata and multi strata measurement results are, in fact, already serialized as JSON structures, you can simply merge them together using JContainer.Merge(Object, JsonMergeSettings) with the merge setting MergeArrayHandling.Concat as follows:

// Get the initial measurement JSON measurements as strings.
IEnumerable<string> measturements = GetJsonMeasurements();

// And concatenate them together into a combined `JObject`:
var settings = new JsonMergeSettings { MergeArrayHandling = MergeArrayHandling.Concat };
var json = measturements.Aggregate(new JObject(),
                                   (j, s) => { j.Merge(JObject.Parse(s), settings); return j; });

Here I am assuming the measurements have already been grouped by "testTIN" value. If not, this is easily added by parsing all the results to a JObject and grouping by "testTIN" value before aggregating as follows:

var settings = new JsonMergeSettings { MergeArrayHandling = MergeArrayHandling.Concat };
var json = measurements
    .Select(j => JObject.Parse(j))
    .GroupBy(j => (string)j["testTIN"])
    .Select(g => g.Aggregate(new JObject(),
                             (j1, j2) => { j1.Merge(j2, settings); return j1; })
            )
    .ToList();

Alternatively, if your JSON results are stored in some collection of files, you can merge directly from the files as follows:

var settings = new JsonMergeSettings { MergeArrayHandling = MergeArrayHandling.Concat };
var json = fileNames.Aggregate(new JObject(),
                               (j, s) => 
                               { 
                                   using (var file = File.OpenText(s))
                                   using (var reader = new JsonTextReader(file))
                                   {
                                       j.Merge(JToken.Load(reader), settings);
                                   }
                                   return j; 
                               });

Demo fiddle with unit tests here.

Update

if you have an enumerable of objects and wish to create a combined JSON file by merging together their JSON representations, you can project each object to a JObject using JObject.FromObject, then merge those:

// Get the results
IEnumerable<MeasurementSet__NoStrata> measurements1 = GetNoStrataMeasurements(); // Get no-strata measurements.
IEnumerable<MeasurementSet__MultiStrata> measurements2 = GetMultiStrataMeasurements(); // Get multistrata measurements.

// Combine them into a single enumerable
IEnumerable<object> measurements = measurements1.Cast<object>()
    .Concat(measurements2.Cast<object>());

// Select serialization and merge settings
var serializerSettings = new JsonSerializerSettings
{
    // Put whatever you want here, e.g.
    NullValueHandling = NullValueHandling.Ignore,
};
var mergeSettings = new JsonMergeSettings 
{ 
    // Required
    MergeArrayHandling = MergeArrayHandling.Concat,
    // Put whatever you want here, either Ignore or Merge
    // https://www.newtonsoft.com/json/help/html/T_Newtonsoft_Json_Linq_MergeNullValueHandling.htm
    MergeNullValueHandling = MergeNullValueHandling.Ignore, // Or Merge if you prefer
};

// Serialize and merge the results
var serializer = JsonSerializer.CreateDefault(serializerSettings);
var json = measurements
    .Select(m => JObject.FromObject(m, serializer))
    .GroupBy(j => (string)j["testTIN"])
    .Select(g => g.Aggregate(new JObject(),
                             (j1, j2) => { j1.Merge(j2, mergeSettings); return j1; })
            )
    // Do we need to remove the `"category"` property?
    // If so do it here.
    .Select(o => { o.Remove("category"); return o; })
    .ToList();

Demo fiddle #2 here.

Well you could use `JToken.FromObject()` to serialize your POCOs directly to objects without an intermediate string representation, then merge those. — dbc, Jan 07 '19 at 05:00
@Lijo - But otherwise this may not be a JSON problem at all, it may be that you need to redesign your domain model, merge `MeasurementSet__NoStrata` and `MeasurementSet__MultiStrata`, and turn `measurements` into a polymorphic collection of measurement types. — dbc, Jan 07 '19 at 05:07

Combine two JSON structures into one in C#

2 Answers2