Laravel Eloquent cursor() out of memory

Question

I have a problem with Laravel's ORM Eloquent cursor() method. Im trying to create some analytics functions for my ecom to get the amount of orders/income and stuff like that, you can check below my code:

$fromDate = Carbon::now()->startOfYear()->toDateString();
$tillDate = Carbon::now()->endOfYear()->toDateString();

$orders = Order::cursor()->whereBetween('created_at', [$fromDate, $tillDate])->whereIn('status', array(1, 2, 3))->filter(function ($order) { 
  return $order; 
});
$countO = $orders->count();
$sumO = $orders->sum('total');

$orders = VoucherOrder::cursor()->whereBetween('created_at', [$fromDate, $tillDate])->where('status', 1)->filter(function ($order) {
  return $order;
});
$countV = $orders->count();
$sumV = $orders->sum('total');

$orders = $countO + $countV;
$income = $sumO + $sumV;

if($orders != 0) {
  $avgOrder = $income / $orders;
}
else {
  $avgOrder = 0;
}

return view('admin.income', compact('orders', 'income', 'avgOrder'));

I used cursor() in order to reduce the memory usage since i'm processing large amounts of data but i still get this error:

Allowed memory size of 536870912 bytes exhausted (tried to allocate 2338688 bytes)

what am i doing wrong? how can i solve?

vist https://stackoverflow.com/questions/34864524/allowed-memory-size-of-536870912-bytes-exhausted-in-laravel — Waleed Muaz, Jul 16 '21 at 10:47

score 1 · Accepted Answer · answered Jul 16 '21 at 10:55

If you want to reduce memory usage without editing php.ini file, I suggest using chunk than cursor().

Model::chunk(250, function ($data) {
    foreach ($d as $data) {
        // your code
    }
});

This will take less memory but do keep in mind this may consume few seconds more for execution in contrast to cursor().

score 0 · Answer 2 · answered Aug 06 '23 at 19:09

I've wrote a blog post about it. But basically, you need to use unbuffered queries if you don't want to have an out of memory.

Blog post: https://medium.com/@JoseCardona/boost-your-apps-performance-processing-large-mysql-tables-with-eloquent-orm-3724421d97aa

score -1 · Answer 3 · answered Jul 16 '21 at 10:49

-1

try editing /etc/php5/fpm/php.ini:

memory_limit = 2048M restart nginx:

sudo systemctl restart nginx

answered Jul 16 '21 at 10:49

Waleed Muaz

737
6
17

any idea on how to solve without increasing memory_limit? – Mono.WTF Jul 16 '21 at 10:50
You can chunk the cache as well if you can't load it all at once. Do you need all the data to be available at the same time? That is a lot of records to load into memory at once. Maybe you can manage to work things out so that you can do what you need to with those records a chunk at a time and remove them from memory when you are done with them. – Waleed Muaz Jul 16 '21 at 10:51
the page shows the income and the amount of orders based on a date range, so ye if i select the whole year there are a lot of data to process – Mono.WTF Jul 16 '21 at 10:58

Laravel Eloquent cursor() out of memory

3 Answers3