C# EF Core build dynamic select statement with expression tree

Question

I want to dynamically create a select statement that creates an array of objects through an array initializer. Those initializers are taken from a provided list of property expressions.

In this example we want to list just the 'Component' property of an entity called 'topic'.

This is how the select statement should look like:

Query.Select(topic => new object[] { topic.Component });

And here is how I create that expression dynamically:

// an example expression to be used. We only need its body: topic.Component
Expression<Func<Topic, object>> providedExpression = topic => topic.Component;
            
// a list of the initializers: new object[] { expression 1, expression 2, ..}. We only use the example expression here
List<Expression> initializers = new List<Expression>() { providedExpression.Body };
            
// the expression: new object[] {...} 
NewArrayExpression newArrayExpression = Expression.NewArrayInit(typeof(object), initializers);

// the expression topic => 
var topicParam = Expression.Parameter(typeof(Topic), "topic"); 
            
// the full expression  topic => new object[] { ... };
Expression<Func<Topic, object[]>> lambda = Expression.Lambda<Func<Topic, object[]>>(newArrayExpression, topicParam);

// pass the expression
Query.Select(lambda);

Now, the created expression looks exactly like the example above, but EF Core throws the good old

The LINQ expression 'topic' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly...

But even from the debugger (see image), the (working) example expression and the generated one are identical. Where does the magic happen that I don't understand? Any tips?

Generated and example expression in debugger

Michael Liu · Accepted Answer · 2022-04-21T21:28:50.707

3

The generated and example expressions may look identical in the debugger, but they actually aren’t. The problem is that your lambda expression references two ParameterExpression objects, both named topic:

The first is created implicitly by the C# compiler when it converts topic => topic.Component to an Expression.
The second, topicParam, is created explicitly.

Even though the two ParameterExpression objects have identical names, they’re treated as distinct parameters. To fix the code, you must ensure that the same ParameterExpression object is used in both the parameter list and body of the lambda:

var topicParam = providedExpression.Parameters[0]; // instead of Expression.Parameter

However, if you have multiple provided expressions, then the C# compiler will generate multiple topic ParameterExpression objects, so this simple fix won’t work. Instead, you will need to replace the auto-generated topic parameter in each providedExpression with your explicitly created ParameterExpression:

public class ParameterSubstituter : ExpressionVisitor
{
    private readonly ParameterExpression _substituteExpression;

    public ParameterSubstituter(ParameterExpression substituteExpression)
    {
        _substituteExpression = substituteExpression;
    }

    protected override Expression VisitParameter(ParameterExpression node)
    {
        return _substituteExpression;
    }
}

And in your method:

var topicParam = Expression.Parameter(typeof(Topic), "topic");
List<Expression> initializers =
    new List<Expression>
    {
        new ParameterSubstituter(topicParam).Visit(providedExpression.Body)
    };
NewArrayExpression newArrayExpression = Expression.NewArrayInit(typeof(object), initializers);
Expression<Func<Topic, object[]>> lambda = Expression.Lambda<Func<Topic, object[]>>(newArrayExpression, topicParam);

edited Apr 21 '22 at 21:28

answered Apr 21 '22 at 21:11

Michael Liu

52,147
13
117
150

Great piece of code that helped me to solve a similar problem in my application! Is it possible to adapt the code to generate an anonymous type "on-the-fly" instead of the object[] array? The query should look like this : Query.Select(topic => new { topic.Component1, topic.Component2, topic.Component3 }); instead of Query.Select(topic => new object[] { topic.Component1, topic.Component2, topic.Component3 }); – RickyTad Aug 11 '23 at 15:53
@RickyTad: You can use Reflection Emit to generate dynamic types, but I don't know whether dynamic types work with LINQ expressions. – Michael Liu Aug 12 '23 at 04:57
Ok, thanks, I will look after some examples with Reflection Emit. What about using a DTO class instead of the object[]? How can the code be adapted to have something like Query.Select(topic => new TopicDTO { Comp1=topic.Component1, Comp2=topic.Component2, Comp3=topic.Component3 }); TopicDTO is a class, something like class TopicDTO { public int Comp1 { get; set; } public int Comp2 { get; set; } public int Comp3 { get; set; } } – RickyTad Aug 12 '23 at 11:05
@RickyTad: Use [Expression.MemberInit](https://learn.microsoft.com/en-us/dotnet/api/system.linq.expressions.expression.memberinit) instead of Expression.NewArrayInit. – Michael Liu Aug 12 '23 at 15:51
Could you please post a code snippet? Actual code: if (addComp1) { Expression> comp1Expr = topic => topic.Comp1; initializers.Add(new ParameterSubstituter(topicExprParam).Visit(comp1Expr.Body)); } if (addComp2) { Expression> comp2Expr = topic => topic.Comp2; initializers.Add(new ParameterSubstituter(topicExprParam).Visit(comp2Expr.Body)); } ... NewArrayExpression newArrExpr = Expression.NewArrayInit(typeof(object), initializers); Expression> lambda = Expression.Lambda>(newArrayExpr, topicExprParam); – RickyTad Aug 12 '23 at 18:35
@RickyTad: Please post a new question. – Michael Liu Aug 13 '23 at 16:01
https://stackoverflow.com/questions/76894950/ef-core-6-dynamically-create-linq-expression-for-choosing-columns-in-select-sta – RickyTad Aug 13 '23 at 19:47

C# EF Core build dynamic select statement with expression tree

1 Answers1

Linked