How can I parse a JSON into a dynamic object allowing access to fields via dot notation using System.Text.Json?

Question

I'm implementing a feature where a user can write an arbitrary expression that needs to be evaluated against some context. Users should be able to write expressions referring to a dynamic JSON payload.

I'm using Roslyn CSharpScript to execute the expression.

I managed to make it work using Newtonsoft, however I would like to do the same using System.Text.Json.

Working approach with Newtonsoft:

    [Fact]
    public async Task Should_Evaluate_Dynamic_Payload_Using_Newtonsoft()
    {
        // Arrange
        const string jsonPayload = """
                                    {
                                        "dog": {
                                            "name": "Slayer"
                                        }
                                    }
                                    """;
        var evaluationContext = new EvaluationContext
        {
            Payload = JObject.Parse(jsonPayload),
        };

        const string expression = @"Payload.dog.name == ""Slayer""";

        var options = ScriptOptions.Default;
        options = options.AddReferences(typeof(CSharpArgumentInfo).Assembly);
        var script = CSharpScript.Create<bool>(expression, options, globalsType: typeof(EvaluationContext));
        var scriptRunner = script.CreateDelegate();

        // Act
        var result = await scriptRunner(evaluationContext);

        // Assert
        result.Should().BeTrue();
    }

Not working approach with System.Text.Json:

    [Fact]
    public async Task Should_Evaluate_Dynamic_Payload_Using_SystemTextJson()
    {
        // Arrange
        const string jsonPayload = """
                                    {
                                        "dog": {
                                            "name": "Slayer"
                                        }
                                    }
                                    """;
        var evaluationContext = new EvaluationContext
        {
            Payload = JsonSerializer.Deserialize<JsonElement>(jsonPayload),
        };

        const string expression = @"Payload.dog.name == ""Slayer""";

        var options = ScriptOptions.Default;
        options = options.AddReferences(typeof(CSharpArgumentInfo).Assembly);
        var script = CSharpScript.Create<bool>(expression, options, globalsType: typeof(EvaluationContext));
        var scriptRunner = script.CreateDelegate();

        // Act
        var result = await scriptRunner(evaluationContext);

        // Assert
        result.Should().BeTrue();
    }

This fails with 'System.Text.Json.JsonElement' does not contain a definition for 'dog'. It seems like the parsed JsonElement is not being correctly identified as a dynamic, hence accessing the property with dot notation fails. I know I could use Payload.GetProperty("dog").GetProperty("name").ToString(), but I'd much prefer keeping the dot notation (Payload.dog.name).

I tried parsing the JSON with JsonDocument, JsonElement, JsonSerializer.Deserialize, but none seem to work.

score 0 · Answer 1 · answered Apr 27 '23 at 11:17

Text.Json using the same syntax as Newtonsoft.Json

var Payload = JsonObject.Parse(jsonPayload);
    
var name = (string) Payload["dog"]["name"];
var path = (string) Payload["dog"]["name"].GetPath(); //$.dog.name

but System.Text.Json supports only get path. IMHO Text.Json still is too raw for using. I highly recommend you to use Newtonsoft.Json. It is not ony one problem, the longer you try to use Text.Json the more problem you have. Sooner or later you will give up.

score 0 · Answer 2 · answered May 03 '23 at 17:18

JToken implements IDynamicMetaObjectProvider which is what allows your Json.NET code to work: it dynamically forwards .NET property references to JSON property dictionary lookups, so that:

string name = ((dynamic)evaluationContext.Payload).dog.name;

Is evaluated in runtime as:

string name = ((JToken)evaluationContext.Payload)["dog"]["name"]?.ToString();

JsonElement and JsonNode, however, do not implement IDynamicMetaObjectProvider, so the above dynamic expression will not work with them. See Json Deserialization of "dynamic" does not work #66638 for confirmation.

What you could do instead is to deserialize your JSON to an ExpandoObject, then pass that into your CSharpScript. To do that, copy ObjectAsPrimitiveConverter from this answer to C# - Deserializing nested json to nested Dictionary<string, object>. Assuming your EvaluationContext type looks like:

public class EvaluationContext { public dynamic Payload { get; set; } }

You can modify your code as follows:

var jsonOptions = new JsonSerializerOptions
{
    Converters = { new ObjectAsPrimitiveConverter(objectFormat : ObjectFormat.Expando, // Use  ObjectFormat.Expando to enable dynamic evaluation
                                                  floatFormat : FloatFormat.Double, unknownNumberFormat : UnknownNumberFormat.Error) },
    WriteIndented = true,
};
var evaluationContext = new EvaluationContext
{
    Payload = JsonSerializer.Deserialize<dynamic>(jsonPayload, jsonOptions),
};

And the remainder should work as-is.

Demo fiddle here.

As an alternative, since your expressions look like JSONPath queries, you might consider using a 3rd party JSONPath evaluator for System.Text.Json (which does not support JSONPath itself, unlike Newtonsoft). There are several mentioned in Add JsonPath support to JsonDocument/JsonElement #31068 including:

How can I parse a JSON into a dynamic object allowing access to fields via dot notation using System.Text.Json?

2 Answers2