1

I using .Net Core & EF Core 3.1

I Have a Expression like Under, but when run throw this exception

could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync(). See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.

my code is

var years = new List<string> { "1390","1391" };
queryable = queryable.Where(x =>
     x.ProdYears != null &&
     years.Any(z => x.ProdYears.Contains(z))
 );

Why this code not working & how can fix this?

Mahdi Radi
  • 429
  • 2
  • 10
  • 30
  • Have you looked at the page linked in the error message? I think that will explain your problem. – Achtung Jan 01 '21 at 22:04
  • 1
    yes, i checked but not useful, it explain "client evaluation" that I don't want use it – Mahdi Radi Jan 01 '21 at 22:09
  • You are using client evaluation (client here is your application and server is the database). EF Core converts your query to sql for example. If it cant convert your query to SQL it will throw an error like this. You either need to refactor to something m,ore eeasy to convert or do your check in memory. – Achtung Jan 01 '21 at 22:12
  • 1
    "client evaluation" mean get all of records from database then filter on that, so has bad performance and i don't want use it – Mahdi Radi Jan 01 '21 at 22:16
  • Thats correct, best is if you rewrite your query so it can be converted to sql. – Achtung Jan 01 '21 at 22:18
  • my friend you missed the point, this query work correctly on EF6, but don't work on EF Core 3.1 and i want to know why and how change my query to work correctly. – Mahdi Radi Jan 01 '21 at 22:21
  • Please show us a [Minimal, Reproducible Example](https://stackoverflow.com/help/minimal-reproducible-example). What is `queryable` and where is `AdVerBuy` property defined? – Hooman Bahreini Jan 01 '21 at 22:54
  • queryable is AsQueryable() from primary table, and AdVerBuy is child table that access to it from main table, actually AdVerBuy not difference to main problem, you can consider query like this `years.Any(z => x.ProdYears.Contains(z))` – Mahdi Radi Jan 01 '21 at 23:06
  • *this query work correctly on EF6* - maybe, but EF team changed their minds on definition of "correctly". Back then it was "if query cannot be translated to sql, then drag all the rows to the client side and evaluate on the client" so your query worked "correctly" and performance was probably terrible/resource waste was large. Now it's "if query cannot be translated, give an error and key developer explicitly decide whether to drag all rows to client" so you have to make the call.. and you said you don't wanna do client eval, so you're now stuck; (but at least efcore gave a choice) – Caius Jard Jan 01 '21 at 23:16
  • thanks, buy `ProdYears` is string and can not convert to AsEnumerable – Mahdi Radi Jan 01 '21 at 23:24
  • I think mostly this pattern "list.Any" is translated to SQL IN, like `persons.Where(p => years.Any(y => p.BirthYear == y))` becomes `WHERE p.BirthYear IN(1390,1391)` but looks like you want EF to write it as `WHERE prodyears LIKE '%1390%' OR prodyears LIKE '%1391%'` ... and I genuinely don't know if that's a case EF can handle.. – Caius Jard Jan 01 '21 at 23:33
  • Perhaps have a think about how you want this query to execute and look at other ways to make EF Write it, such as with some helper way of hinting to EF https://stackoverflow.com/questions/45708715/entity-framework-ef-functions-like-vs-string-contains - or post more of your data model and example db content, entities etc so we can think of ways to make EF perform a translation it does know how to do – Caius Jard Jan 01 '21 at 23:36
  • May you can dynamically build an Expression with repeated OrElse - https://stackoverflow.com/questions/56197261/c-sharp-linq-combine-multiple-where-with-an-or-clause/56199758 - or maybe all this is because ProdYears is some CSV data stored in a row and someone said way back when "don't do it; it'll be a pain in the ass one day" and someone else went "meh, it's future guy's problem".. and you're future guy-> can the problem be resolved by properly refactoring the data storage instead?? – Caius Jard Jan 01 '21 at 23:48
  • Of course the root of your problem is that `AdVerBuy.ProdYears` isn't normalized. – Gert Arnold Jan 02 '21 at 10:25
  • @CaiusJard That isn't true of EF 6.x - it just could translate more forms than EF Core - EF Core 2.x was the one that did automatic client side processing. LINQ to SQL / EF have a lot more translation capability than EF Core 3.x / 5.x, they have deliberately decided not to support general LINQ query translation. – NetMage Jan 04 '21 at 19:41

2 Answers2

1

You can return all match items from the list using this extension method I found and modified from: Reference

This is not generic, so you will have to change "ContainTest" to your entity type.

    public static class EntityHelper
    {
        public static IQueryable<ContainTest> SqlLikeInList(this IQueryable<ContainTest> products, List<string> containsList)
        {
            if (!containsList.Any()) return null;

            var patterns = containsList.Select(t => $"%{t}%").ToList();

            ParameterExpression parameter = Expression.Parameter(typeof(ContainTest));
            Expression body = patterns.Select(word => Expression.Call(typeof(DbFunctionsExtensions).GetMethod(nameof(DbFunctionsExtensions.Like),
                                                                      new[]{typeof(DbFunctions), typeof(string), typeof(string)}),
                                                                      Expression.Constant(EF.Functions),
                                                                      Expression.Property(parameter, typeof(ContainTest).GetProperty(nameof(ContainTest.ProdYears))),
                                                                      Expression.Constant(word)))
                                      .Aggregate<MethodCallExpression, Expression>(null, (current, call) => current != null ? Expression.OrElse(current, call) : (Expression)call);

            return products.Where(Expression.Lambda<Func<ContainTest, bool>>(body, parameter));
        }

and then call it from your DBSet like this:

var years = new List<string> { "1390", "1391" };
IQueryable<ContainTest> queryable = context.ContainTests.SqlLikeInList(years);

From testing, I found that all of that could be avoided if you were looking for exact matches of '1390' or '1391'

years.Any(z => x.ProdYears == z)

...so it is the .contains statement that was forcing you to use client side evaluation. Using this extension method, however, allows you to use a single call for server side evaluation.

Using SQL Server Profiler, my example produces this SQL

SELECT [c].[ContainTestId], [c].[ProdYears], [c].[RequiredContentColumn]
FROM [ContainTests] AS [c]
WHERE ([c].[ProdYears] LIKE N'%1390%') OR ([c].[ProdYears] LIKE N'%1391%')

For reference, I created an entity to recreate your scenario as such (which is what the SQL references)

    public class ContainTest
    {
        public long ContainTestId { get; set; }
        public string? ProdYears { get; set; }
        public string RequiredContentColumn { get; set; }
    }

and I populated the table with this sample data:

SET IDENTITY_INSERT [dbo].[ContainTests] ON
INSERT INTO [dbo].[ContainTests] ([ContainTestId], [ProdYears], [RequiredContentColumn]) VALUES (1, N'aa1389zz', N'dont get me')
INSERT INTO [dbo].[ContainTests] ([ContainTestId], [ProdYears], [RequiredContentColumn]) VALUES (2, N'zz1390aa', N'get me')
INSERT INTO [dbo].[ContainTests] ([ContainTestId], [ProdYears], [RequiredContentColumn]) VALUES (3, N'aa1391zz', N'get me too')
INSERT INTO [dbo].[ContainTests] ([ContainTestId], [ProdYears], [RequiredContentColumn]) VALUES (4, NULL, N'dont get me')
SET IDENTITY_INSERT [dbo].[ContainTests] OFF

Happy Coding!!!

CANDIMAN
  • 119
  • 1
  • 5
  • Extremely well written answer, and works perfectly. I made mine generic and added a string parameter for target property name. 100% works on efcore v5. Thank you!! :) – Barry Mar 21 '22 at 23:47
0

If you are willing to use LINQKit, you can create extension methods to handle your query:

public static class LinqKitExt { // using LINQKit
    // searchTerms - IEnumerable<TSearch> where one must match for a row
    // testFne(row,searchTerm) - test one of searchTerms against a row
    // r => searchTerms.All(s => testFne(r,s))
    public static Expression<Func<T, bool>> AnyIs<T, TSearch>(this IEnumerable<TSearch> searchTerms, Expression<Func<T, TSearch, bool>> testFne) {
        var pred = PredicateBuilder.New<T>();
        foreach (var s in searchTerms)
            pred = pred.Or(r => testFne.Invoke(r, s));
    
        return pred;
    }

    // searchTerms - IEnumerable<TSearch> where one must match for a row
    // testFne(row,searchTerm) - test one of searchTerms against a row
    // dbq.Where(r => searchTerms.Any(s => testFne(r,s)))
    public static IQueryable<T> WhereAny<T,TSearch>(this IQueryable<T> dbq, IEnumerable<TSearch> searchTerms, Expression<Func<T, TSearch, bool>> testFne) =>
        dbq.AsExpandable().Where(searchTerms.AnyIs(testFne));
    public static IQueryable<T> WhereAny<T,TSearch>(this IQueryable<T> dbq, Expression<Func<T, TSearch, bool>> testFne, IEnumerable<TSearch> searchTerms) =>
        dbq.AsExpandable().Where(searchTerms.AnyIs(testFne));
    public static IQueryable<T> WhereAny<T,TSearch>(this IQueryable<T> dbq, Expression<Func<T, TSearch, bool>> testFne, params TSearch[] searchTerms) =>
        dbq.AsExpandable().Where(searchTerms.AnyIs(testFne));
}

Then you can use the methods:

var years = new List<string> { "1390","1391" };
queryable = queryable.WhereAny(years, (qr, y) => qr.ProdYears.Contains(y));

NOTE: You don't need to test against null as SQL will handle it automatically.

NetMage
  • 26,163
  • 3
  • 34
  • 55