How to select top N rows for each group in a Entity Framework GroupBy with EF 3.1

Question

I need to get top 10 rows for each group in a table with entity framework. Based on other solution on SO, I tried 2 things:

var sendDocuments = await context.Set<DbDocument>
    .Where(t => partnerIds.Contains(t.SenderId))
    .GroupBy(t => t.SenderId)
    .Select(t => new
    {
        t.Key,
        Documents = t.OrderByDescending(t2 => t2.InsertedDateTime).Take(10)
    })                
    .ToArrayAsync();

error:

System.InvalidOperationException: 'The LINQ expression
'(GroupByShaperExpression: KeySelector: (d.SenderId), 
ElementSelector:(EntityShaperExpression: 
    EntityType: DbDocument
    ValueBufferExpression: 
        (ProjectionBindingExpression: EmptyProjectionMember)
    IsNullable: False ) )
    .OrderByDescending(t2 => t2.InsertedDateTime)' could not be translated. Either rewrite the query in a form that can be translated,
> or switch to client evaluation explicitly by inserting a call to
> either AsEnumerable(), AsAsyncEnumerable(), ToList(), or
> ToListAsync().

and

var sendDocuments2 = await context.Set<DbDocument>
    .Where(t => partnerIds.Contains(t.SenderId))
    .GroupBy(t => t.SenderId)
    .SelectMany(t => t.OrderByDescending(t2 => t2.InsertedDateTime).Take(10))
    .ToArrayAsync();

error:

System.InvalidOperationException: 'Processing of the LINQ expression 't => t .OrderByDescending(t2 => t2.InsertedDateTime) .AsQueryable() .Take(10)' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core.

Any other idea?

can you provide what you want to do with an sql query? or can you provide the table structure? — Asım Gündüz, Dec 23 '19 at 15:03

Ivan Stoev · Accepted Answer · 2021-11-10T07:22:45.727

Update (EF Core 6.0):

EF Core 6.0 added support for translating GroupBy result set projection, so the original code for taking (key, items) now works as it should, i.e.

var query = context.Set<DbDocument>()
    .Where(e => partnerIds.Contains(e.SenderId))
    .GroupBy(e => e.SenderId)
    .Select(g => new
    {
        g.Key,
        Documents = g.OrderByDescending(e => e.InsertedDateTime).Take(10)
    });

However flattening (via SelectMany) is still unsupported, so you have to use the below workaround if you need such query shape.

Original (EF Core 3.0/3.1/5.0):

This is a common problem, unfortunately not supported by EF Core 3.0/3.1/5.0 query translator specifically for GroupBy.

The workaround is to do the groping manually by correlating 2 subqueries - one for keys and one for corresponding data.

Applying it to your examples would be something like this.

If you need (key, items) pairs:

var query = context.Set<DbDocument>()
    .Where(t => partnerIds.Contains(t.SenderId))
    .Select(t => t.SenderId).Distinct() // <--
    .Select(key => new
    {
        Key = key,
        Documents = 
            context.Set<DbDocument>().Where(t => t.SenderId == key) // <--
                 .OrderByDescending(t => t.InsertedDateTime).Take(10)
                 .ToList() // <--
    });

If you need just flat result set containing top N items per key:

var query = context.Set<DbDocument>()
    .Where(t => partnerIds.Contains(t.SenderId))
    .Select(t => t.SenderId).Distinct() // <--
    .SelectMany(key => context.Set<DbDocument>().Where(t => t.SenderId == key) // <--
        .OrderByDescending(t => t.InsertedDateTime).Take(10)
    );

First query doesn't work when there are multiple key. It is always getting one key. — Murat Can OĞUZHAN, Jan 13 '20 at 17:30
This helped me in so many ways that I cannot explain. When doing a GroupBy(x => x.key), translate it to Select(x => x.key).Distinct().SelectMany(key => .Where(t => t.key == key). — Carlos M. Hernández, May 21 '20 at 17:55

How to select top N rows for each group in a Entity Framework GroupBy with EF 3.1

1 Answers1

Linked

Related