-1

I would like to ask you how to avoid error connected with Dynamic Invoke in LINQ expression.

Existing error:

Method 'System.Object DynamicInvoke(System.Object[])' has no supported translation to SQL

Example of code (LINQPad):

Edited:

void Main()
{
var entity = new[] 
{     
    new Customers    {  CustomerID = "ALFKI", CompanyName = "Alfreds Futterkiste", ContactName = "Maria Anders"},
    new Customers    {  CustomerID = "ANATR", CompanyName = "Ana Trujillo Emparedados y helados", ContactName = "Ana Trujillo"},
    new Customers    {  CustomerID = "2", CompanyName = "Louis Vuiton2", ContactName = "Dom2"}};

    var result = Exists(entity, x => x.CustomerID + " " + x.CompanyName); 
}


IDictionary<string, bool> Exists(IEnumerable<Customers> data, Func<Customers, object> predicate)
{  
   Expression<Func<Customers, object>> expression = x => predicate(x);
   var ids = data.AsQueryable().Select(x=>predicate(x));        
   var existing = Customers.Where(x => ids.Contains(expression)).ToList(); //Error line.

   //I do not want to materialize query after Customers for example Customers.ToList()[...]

   data.ToList().ForEach(x => existing.Any(y=> predicate(y).ToString() == predicate(x).ToString()));
   var dictionary = data.ToDictionary(x => x.CustomerID.ToString(), x => existing.Any(y => predicate(y).ToString() == predicate(x).ToString()));               

   return dictionary; 
}   

Edit: var existing should return:

enter image description here

but return null.

Liam
  • 27,717
  • 28
  • 128
  • 190
  • use `Expression>` instead of `Func` - https://stackoverflow.com/questions/793571/why-would-you-use-expressionfunct-rather-than-funct – Rand Random Aug 27 '20 at 14:25
  • Still does not work. Maybe now not returns error but return null. – expressiontree Aug 27 '20 at 14:34
  • What is the actual goal with the method? What output would you expect? – JonasH Aug 27 '20 at 14:42
  • Instead of stringifying everything you would be better off creating a compound `Expression>` that does what you are trying to do. – Ian Mercer Aug 27 '20 at 14:43
  • Pls try to remove `.AsQueryable()` from this line `var ids = data.AsQueryable().Select(x=>predicate(x));` - I believe it's not needed, but don't know if that is the problem. – Rand Random Aug 27 '20 at 14:44
  • Problem appears in " var existing" line only if i not materialize query (i do not do that for the sake of performance). – expressiontree Aug 27 '20 at 14:47
  • 1
    @expressiontree The error complains about an attempt to use EF or EF Core with an expression that can't be converted to SQL. The code posted here has nothing to do with EF Core though and doesn't help. What is the *actual* code that throws this error? And why would a database query contain `DynamicInvoke`? – Panagiotis Kanavos Aug 27 '20 at 14:50
  • But `data` is already materialized, you don't want `Customers` to be materialized, which shouldn't change not matter if you would `data` to be `.AsQueryable()` or not, for example here: https://stackoverflow.com/questions/21641016/check-if-list-contains-item-from-other-list-in-entityframework - which just uses a simple list and this will be translated to a `IN` statement on the server. – Rand Random Aug 27 '20 at 14:51
  • @Panagiotis Kanavos So do you know how to solve this problem? – expressiontree Aug 27 '20 at 14:51
  • What problem? You didn't post the code that throws. To fix it, don't use code that can't be translated to SQL. `DynamicInvoke` makes no sense in a SQL query, it's a purely local function against an object – Panagiotis Kanavos Aug 27 '20 at 14:51
  • @Rand Random But Customers is from database (11 mln records) but ids (maybe 2 records). – expressiontree Aug 27 '20 at 14:52
  • @expressiontree the reason this question is getting downvoted is because it doesn't contain any relevant info apart from `has no supported translation to SQL`. EF and EF Core *don't* use that method, so it's your own code that does this. How, where, we can't know – Panagiotis Kanavos Aug 27 '20 at 14:53
  • Yeah, but as I am trying to say even if you would remove `AsQueryable()` from `data` it wouldn't change the behaviour of `Customers` – Rand Random Aug 27 '20 at 14:53
  • @ Rand Random it does not matter i can also change to your code. The most important is line with var existing. – expressiontree Aug 27 '20 at 14:54
  • @expressiontree none of those lines is important because none of them have anything to do with EF. You haven't posted any code that demonstrates a problem yet. What you posted is an *Entity Framework error only*. It's not generated by dictionaries and enumerables, only EF. You haven't even posted the full exception text - the stack trace would tell you which method calls caused the problem and where. – Panagiotis Kanavos Aug 27 '20 at 14:55
  • Yeah, I know, but you said after changing from `Func` to `Expression` the exception is gone and you are now getting an empty result, so I am trying to help with that, I wouldn't know a reason why the line with `var existing` would result in an empty result so the condition must always return false, and the condition is filled with `ids` so maybe changing this will give good results – Rand Random Aug 27 '20 at 14:56
  • Ok, i checked it. With or without AsQueryable i get same result. – expressiontree Aug 27 '20 at 15:00
  • To bad, the only idea I would have that maybe your string comparison is wrong. eg. comparing upper cases with lower cases. Maybe it would help to look what SQL got generated (if any at all) and run this generated SQL directly on your server and look why it doesn't return anything. - https://learn.microsoft.com/en-us/ef/core/miscellaneous/logging – Rand Random Aug 27 '20 at 15:06

1 Answers1

3

Here's how to do it avoiding any nasty conversion to strings. What you really want is to take your array and convert it to Expressions that can be applied to Customer objects and then aggregate those using Expression.OrElse to create a single Query expression that can then be applied to the database.

It's not simple, but here's how to do that.

You'd call the method at the end like so:

var result = Exists(Customers.AsQueryable(), 
    entity, 
    (q) => c => (q.CustomerID == c.CustomerID && q.CompanyName == c.CompanyName)); 

This has several advantages over 'stringfying' everything in your comparison. For one, the database can optimize the query using indices. For another you can pass more complex expressions that simple string comparisons if you wish, e.g. c.CustomerID > q.CustomerID.

I've separated a CustomerQuery class from the CustomerClass because they are different (and fixed your pluralization).

The actual method that does the work is quite simple. All the methods before that are for rewriting Expressions with different parameters to create the OrElse expression that you want to create. These methods are generally useful any time you want to manipulate expressions and understanding the base ExpressionVisitor class and how parameter substitution works is a useful excercise. Note how it takes a Func that maps a CustomerQuery into an Expression that can be applied to the Customer database.

/// <summary>
/// An ExpressionVisitor for parameter substitution
/// </summary>
internal class ExpressionParameterSubstitute : ExpressionVisitor
{
    private readonly ParameterExpression from;
    private readonly Expression to;

    /// <summary>
    /// Creates a new instance of the <see cref="ExpressionParameterSubstitute"/> visitor
    /// </summary>
    public ExpressionParameterSubstitute(ParameterExpression from, Expression to)
    {
        this.from = from;
        this.to = to;
    }

    /// <summary>
    /// Visit a Lambda Expression
    /// </summary>
    protected override Expression VisitLambda<T>(Expression<T> node)
    {
        if (node.Parameters.All(p => p != this.from))
            return node;

        // We need to replace the `from` parameter, but in its place we need the `to` parameter(s)
        // e.g. F<DateTime,Bool> subst F<Source,DateTime> => F<Source,bool>
        // e.g. F<DateTime,Bool> subst F<Source1,Source2,DateTime> => F<Source1,Source2,bool>

        if (to is LambdaExpression toLambda)
        {
            var substituteParameters = toLambda?.Parameters ?? Enumerable.Empty<ParameterExpression>();

            ReadOnlyCollection<ParameterExpression> substitutedParameters
                = new ReadOnlyCollection<ParameterExpression>(node.Parameters
                    .SelectMany(p => p == this.from ? substituteParameters : Enumerable.Repeat(p, 1))
                    .ToList());

            var updatedBody = this.Visit(node.Body); // which will convert parameters to 'to'
            return Expression.Lambda(updatedBody, substitutedParameters);
        }
        else
        {
            // to is not a lambda expression so simple substitution can work
            ReadOnlyCollection<ParameterExpression> substitutedParameters
                = new ReadOnlyCollection<ParameterExpression>(node.Parameters
                    .Where(p => p != this.from)
                    .ToList());

            var updatedBody = this.Visit(node.Body); // which will convert parameters to 'to'

            if (substitutedParameters.Any())
                return Expression.Lambda(updatedBody, substitutedParameters);
            else
                return updatedBody;
        }
    }

    /// <summary>
    /// Visit a ParameterExpression
    /// </summary>
    protected override Expression VisitParameter(ParameterExpression node)
    {
        var toLambda = to as LambdaExpression;
        if (node == from) return toLambda?.Body ?? to;
        return base.VisitParameter(node);
    }
}

 public static Expression<Func<T, bool>> OrElse<T>(
    Expression<Func<T, bool>> expr1,
    Expression<Func<T, bool>> expr2)
{
    var parameter = Expression.Parameter(typeof (T));

    var leftVisitor = new ExpressionParameterSubstitute(expr1.Parameters[0], parameter);
    var left = leftVisitor.Visit(expr1.Body);

    var rightVisitor = new ExpressionParameterSubstitute(expr2.Parameters[0], parameter);
    var right = rightVisitor.Visit(expr2.Body);

    return Expression.Lambda<Func<T, bool>>(
        Expression.OrElse(left, right), parameter);
}

public static IDictionary<string, bool> Exists(IQueryable<Customer> customers, IEnumerable<CustomerQuery> data, Func<CustomerQuery, Expression<Func<Customer, bool>>> predicate)
{  
   Expression<Func<Customer, bool>> expression = x => false;
   foreach (var item in data)
   {
       var exprForOne = predicate.Invoke(item);
       expression = OrElse(expression, exprForOne);
   }

   var split = customers.GroupBy(expression).SelectMany(g => g.Select(c => new {c, g.Key})).ToDictionary(x => x.c.CustomerID, x => x.Key);
   return split;
}   
Ian Mercer
  • 38,490
  • 8
  • 97
  • 133
  • Thank you for your time and greate solution but to be honest I looked for a solution which makes "include" operation instead of "or".The reason was related to efficiency (performance). – expressiontree Aug 28 '20 at 12:08
  • @expressiontree if it was a simple `id` field the `include` approach would be great, but as soon as you combine two fields in an expression like that you are probably best of delegating to SQL to figure out the optimized query to run using indices etc.. the MSFT SQL Database optimizer is amazingly good. – Ian Mercer Aug 28 '20 at 19:18