Apply eloquent eager limit query using SQL ROW_NUMBER in Laravel

Question

As @JonasStaudenmeir answered on laravel eager loading with limit, which query looks like:

User::select('id')
    ->with([
        'posts' => fn($query) => $query->select(['id', 'user_id'])->limit(4)
    ])
    ->limit(2)
->get();

select `id` from `users` limit 2

select * from (select `id`, `user_id`, row_number() over (partition by `posts`.`user_id`) as laravel_row from `posts` where `posts`.`user_id` in (1, 3)) as laravel_table where laravel_row <= 4 order by laravel_row

Now, my interest is to do it manually which is what I tried here:

User::select('id')
->with([
    'posts' => fn($query) => $query->select(['id', 'user_id'])
        ->selectRaw("row_number() over (partition by `posts`.`user_id`) as laravel_row")
        ->where('laravel_row', '<=', 4)
        ->orderBy('laravel_row')
])
->limit(2)
->get();

Also, I take some help from online (SQLtoEloquent), but the syntax was not properly formed so it failed there too.

SQLSTATE[42S22]: Column not found: 1054 Unknown column 'laravel_row' in 'where clause'

select `id`, `user_id`, row_number() over (partition by `posts`.`user_id`) as laravel_row from `posts` where `posts`.`user_id` in (1, 3) and `laravel_row` <= 4 order by `laravel_row` asc

UPDATE:

Following @Igor's advice, I tried to make it similar to the package and more convenient but didn't match the expected output.

App\Providers\AppServiceProvider.php

public function boot(): void
{
    \Illuminate\Database\Eloquent\Relations\Relation::macro('limit', function(int $value) {

        if($this->parent->exists) {

           $this->query->limit($value);

        } elseif ($value >= 0) {

            // When I tried on the Model(User), it succeeded the below logic
            // 2 was expected because the total number of users fetched is 2 but it didn't happen
            $parentLimitValue = $this->query->getQuery()->limit; // null

            // $parentLimitValue = $this->parent->getQuery()->limit; // null
            // $parentLimitValue = $this->related->getQuery()->limit; // null

            $parentLimitValue ??= 1;

            $this->query
                ->selectRaw("row_number() over (partition by ".$this->getExistenceCompareKey().") as laravel_row")
                ->orderBy('laravel_row')
            ->limit($value * ($parentLimitValue ?: 1));
        }
        return $this;
    });
}

Does anyone know where I should put my eyes to minimize this package?

Igor · Accepted Answer · 2023-07-23T18:19:51.957

2

I think you need to remove where and add limit into the subquery

User::select('id')
->with([
    'posts' => fn($query) => $query->select(['id', 'user_id'])
        ->selectRaw("row_number() over (partition by `posts`.`user_id`) as laravel_row")
        ->orderBy('laravel_row')
        ->limit(4) 
])
->limit(2)
->get();

UPDATE

To limit users and posts per users use this:

$usersCount = 2;
$postsPerUser = 4;
$users = User::select('id')
->with([
    'posts' => fn($query) => $query->select(['id', 'user_id'])
        ->selectRaw("row_number() over (partition by `posts`.`user_id`) as laravel_row")
        ->limit($usersCount * $postsPerUser) 
        ->orderBy('laravel_row')
])
->limit($usersCount)
->get();

edited Jul 23 '23 at 18:19

answered Jul 23 '23 at 16:12

Igor

171
1
8

Yeah! But the question is how the data be arranged/taken. Instead of getting 4 posts for each user, it will get 2-2 posts for each user. So, it's not what I expected. – JS TECH Jul 23 '23 at 16:22
Try with ... **->groupBy('user_id')->limit(4)** ... into the subquery? – Igor Jul 23 '23 at 16:28
OK then, seems like is better to create **latestPost** hasOne relation and do it like that. **Partition by** is not seems appropriate for your case. [https://stackoverflow.com/a/33780740/3955714](https://stackoverflow.com/a/33780740/3955714) – Igor Jul 23 '23 at 17:09
1

One possible solution is that we can trick your answer in such ways that it will multiply both limit values i.e. `limit(4x2)`. By doing this, it will match my expected output but it goes query like `limit(8)` at the end in SQL. – JS TECH Jul 23 '23 at 17:15
Also, your groupBy method throws me this error. [Window function is allowed only in SELECT list and ORDER BY clause](https://stackoverflow.com/questions/14111321/windowed-functions-can-only-appear-in-the-select-or-order-by-clauses). If you can answer, please let know? – JS TECH Jul 23 '23 at 17:18

JS TECH · Answer 2 · 2023-07-26T03:25:28.163

As per @Igor's response, I made it more handy by placing it in traits & local scopes.

App\Traits\WithEagerLimit.php

<?php

namespace App\Traits;

use Illuminate\Database\Eloquent\Builder;

trait WithEagerLimit
{
    public function scopeWithEagerLimit(Builder $builder, string $relation, callable $callback)
    {
        return $builder->with([ $relation => function($query) use ($builder, $callback) {

            $limit = $builder->getQuery()->limit ?: 1;

            // Illuminate\Database\Eloquent\Relations\Relation
            $query = call_user_func_array($callback, [$query]);

            // Illuminate\Database\Query\Builder
            $dbQB = $query->getQuery()->getQuery();

            if(!$dbQB->limit) {
                return $query;
            }

            $dbQB->limit *=  $limit;

            return $query->when(is_null($dbQB->columns), fn($q) => $q->select('*'))
                ->selectRaw("row_number() over (partition by ".$query->getExistenceCompareKey().") as laravel_row")
                ->orderBy('laravel_row');
        }]);
    }
}

Use the WithEagerLimit trait on the corresponding model.

For example:

class User extends Model {

    use \App\Traits\WithEagerLimit;
    
    //...

    public function posts()
    {
        return $this->hasMany(Post::class);
    }
}

Now, you can chain withEagerLimit() to any of your queries by doing the following:

Simple eager loading functionality

User::withEagerLimit('posts', fn($query) => $query)->get();

// SQL
select * from `users`

select * from `posts` where `posts`.`user_id` in (1, 2, 3, 4, 5, 6, 7, 8, 9, 10)

Conditions on eager loading functionality

User::query()
    ->withEagerLimit('posts', fn($query) => $query->select(['id', 'user_id'])->where('id', '<=', 5))
->get();

// SQL
select * from `users`

select `id`, `user_id` from `posts` where `posts`.`user_id` in (1, 2, 3, 4, 5, 6, 7, 8, 9, 10) and `id` <= 5

Complex SQL RAW query on eager loading functionality

User::query()
    ->select('id')
    ->withEagerLimit('posts',
        fn($query) => $query->select(['id', 'user_id'])
            ->whereRaw('MOD(id, 2) = 0')
            ->limit(4)
    )
    ->whereRaw('MOD(id, 2) = 1')
    ->skip(3)
    ->take(2)
    ->latest('id')
->get();

// SQL
select `id` from `users` where MOD(id, 2) = 1 order by `id` desc limit 2 offset 3

select `id`, `user_id`, row_number() over (partition by posts.user_id) as laravel_row from `posts` where `posts`.`user_id` in (1, 3) and MOD(id, 2) = 0 order by `laravel_row` asc limit 8

Pagination on eager loading functionality

User::query()
    ->withEagerLimit('posts', fn($query) => $query->limit(4))
->paginate(5);

// SQL
select count(*) as aggregate from `users`

select * from `users` limit 5 offset 0

select *, row_number() over (partition by posts.user_id) as laravel_row from `posts` where `posts`.`user_id` in (1, 2, 3, 4, 5) order by `laravel_row` asc limit 20

For those eagerly waiting for an answer

You should call ->limit($value) or an equivalent method (i.e. take(), skip(), paginate(), or similar procedure) in any queries chained with ->withEagerLimit($relationName, $callback) in the main-query and also in sub-query.

User::query()
    ->select('id')
    ->withEagerLimit('posts', fn($query) => $query->select(['id', 'user_id'])->limit(4))
    ->limit(2)
->get();

You get the benefits of using this trait only when the query meets enough results on both the tables i.e. users & posts. Otherwise, you will get extra results out of the box because here we have done UsersLimit x PostsLimit, so keep the query accordingly.

Apply eloquent eager limit query using SQL ROW_NUMBER in Laravel

2 Answers2