6

SQL Server becomes very slow with Skip/Take on large tables (> 1000000 rows). The tables key column type is Guid and I know the last read row. I try to load next page like

var keyGuid = Guid.NewGuid(); // Key Guid of the last read row
// var result1 = DbContext.Entity.Where(x => x.Id > keyGuid).Take(10).ToList();
var result2 = DbContext.Entity.Where(x => x.Id.CompareTo(keyGuid) > 0).Take(10).ToList();

While the first approach doesn't compile, the second one evaluates the query on client (QueryClientEvaluationWarning) and isn't useful too.

Unfortunately, I cannot modify the database in any way.

Is there any 'native' EF Core solution without custom SQL? It might be ok if it's possible to intercept SQL code generation and resolve the expression manually (but how?)

YosiFZ
  • 7,792
  • 21
  • 114
  • 221
Florian
  • 467
  • 4
  • 8
  • You could write the SQL queries yourself with EF Core like here: https://learn.microsoft.com/en-us/ef/core/querying/raw-sql or http://www.entityframeworktutorial.net/EntityFramework4.3/raw-sql-query-in-entity-framework.aspx – Calin Vlasin Feb 28 '19 at 07:55
  • Thanks, this would be my last alternative. It's a generic component working on any table including some contains compound keys. Not as easy because a bit more custom sql code generation is required. – Florian Feb 28 '19 at 08:21
  • @Florian your code won't work without an `OrderBy()` clause. There's no implicit order in query results unless an `ORDER BY` clause is specified. Even if `ID` is a clustered index, joins and parallel execution can change the way results appear. – Panagiotis Kanavos Feb 28 '19 at 15:28

2 Answers2

18

Update: Guid.CompareTo(Guid) translation is supported in recent EF Core versions, so while the below approach still works and IMHO is more readable and less error prone (one can easily fall into Guid.CompareTo(object) trap which isn't and won't ever be translated), technically it is redundant.


EF Core 2.x:

Starting with v2.0, EF Core supports the so called Database scalar function mapping. It's not very well documented and usually is used to map some database function. But fluent API also allows you to provide a custom translation via HasTranslation method:

Sets a callback that will be invoked to perform custom translation of this function. The callback takes a collection of expressions corresponding to the parameters passed to the function call. The callback should return an expression representing the desired translation.

The following class utilizes that by defining several custom extension methods for comparing Guid values and registers a custom translation for them, which converts the method call expressions to binary comparison expressions, basically simulating the missing >, >=, < and <= Guid operators, which allows translating them to SQL and properly execute server side, as soon as the database supports them (SqlServer does).

Here is the implementation:

public static class GuidFunctions
{
    public static bool IsGreaterThan(this Guid left, Guid right) => left.CompareTo(right) > 0;
    public static bool IsGreaterThanOrEqual(this Guid left, Guid right) => left.CompareTo(right) >= 0;
    public static bool IsLessThan(this Guid left, Guid right) => left.CompareTo(right) < 0;
    public static bool IsLessThanOrEqual(this Guid left, Guid right) => left.CompareTo(right) <= 0;
    public static void Register(ModelBuilder modelBuilder)
    {
        RegisterFunction(modelBuilder, nameof(IsGreaterThan), ExpressionType.GreaterThan);
        RegisterFunction(modelBuilder, nameof(IsGreaterThanOrEqual), ExpressionType.GreaterThanOrEqual);
        RegisterFunction(modelBuilder, nameof(IsLessThan), ExpressionType.LessThan);
        RegisterFunction(modelBuilder, nameof(IsLessThanOrEqual), ExpressionType.LessThanOrEqual);
    }
    static void RegisterFunction(ModelBuilder modelBuilder, string name, ExpressionType type)
    {
        var method = typeof(GuidFunctions).GetMethod(name, new[] { typeof(Guid), typeof(Guid) });
        modelBuilder.HasDbFunction(method).HasTranslation(parameters =>
        {
            var left = parameters.ElementAt(0);
            var right = parameters.ElementAt(1);
            return Expression.MakeBinary(type, left, right, false, method);
        });
    }
}

All you need is to add the following line to your context OnModelCreating override:

GuidFunctions.Register(modelBuilder);

and then simply use them in your queries:

var result = DbContext.Entity
    .Where(x => x.Id.IsGreaterThan(keyGuid))
    .Take(10).ToList();

EF Core 3.0:

HasTranslation now receives and returns SqlExpression instances, so

return Expression.MakeBinary(type, left, right, false, method);

should be replaced with

return new SqlBinaryExpression(type, left, right, typeof(bool), null);
Ivan Stoev
  • 195,425
  • 15
  • 312
  • 343
1

EF 7

The CompareTo method does work and produces the indented sql query.

var result2 = DbContext.Entity
  .Where(x => x.Id.CompareTo(keyGuid) > 0)
  .OrderBy(x => x.Id)
  .Take(10)
  .ToList();
  • Actually, it's supported as of v 3.1, although later the SQL translation got a bit better. And, I guess "intended"? – Gert Arnold Jun 04 '23 at 18:28
  • This page was one of the first found. This Guid.CompareTo method is not obvios. It's the first time that I used anywhere on a Guid. `x.Id > keyGuid` is not possible. – Andreas Dirnberger Jun 06 '23 at 15:47