21

For processing large database, laravel provides chunk method as here https://laravel.com/docs/5.1/queries#retrieving-results

But how can I use chunk method in this query,

 $data = Inspector::latest('id')
                ->select('id', 'firstname', 'status', 'state', 'phone')
                ->where('firstname', 'LIKE', '%' . $searchtext . '%')
                ->get();

where I'm returning json response like this,

echo json_encode($data);

Any suggestions....

Aamir
  • 2,173
  • 1
  • 29
  • 58

5 Answers5

47

As I understand it the chunk() method is for use when you need to work with a large dataset and take an action on that data chunk by chunk.

From your question, it sounds like you're performing a query then returning the data as JSON so to me, it doesn't sound like you're taking an action on your dataset that requires chunking.

If you want to break up the returned JSON data you should be instead looking at pagination.

You could apply pagination to your query like so:

$data = Inspector::latest('id')
    ->select('id', 'firstname', 'status', 'state', 'phone')
    ->where('firstname', 'LIKE', '%' . $searchtext . '%')
    ->paginate();

You can specify the size of each set by passing a number to the paginate method:

$data = Inspector::latest('id')
    ->select('id', 'firstname', 'status', 'state', 'phone')
    ->where('firstname', 'LIKE', '%' . $searchtext . '%')
    ->paginate(25);

If I've misunderstood and you did actually want to do the chunking, I believe you could do the following:

$data = Inspector::latest('id')
    ->select('id', 'firstname', 'status', 'state', 'phone')
    ->where('firstname', 'LIKE', '%' . $searchtext . '%')
    ->chunk(50, function($inspectors) {
        foreach ($inspectors as $inspector) {
            // apply some action to the chunked results here
        }
    });

Also, if you're returning an eloquent object it will be automatically cast to json so you don't need to perform json_encode() as far as I'm aware.

EDIT

If I've completely misunderstood you and what you actually want to do is this:

{ 1000 records } -> this is the result of your query

split it into this:

{
    { 300 records},
    { 300 records},
    { 300 records},
    { 100 records},
}

Then you want the Collection's chunk method:

$data = Inspector::latest('id')
    ->select('id', 'firstname', 'status', 'state', 'phone')
    ->where('firstname', 'LIKE', '%' . $searchtext . '%')
    ->get() // now we're working with a collection
    ->chunk(300);

Remember you're not working with a Collection until you get the result of the query so if you just call chunk() it will expect a callback, which will be applied to the entire dataset, as you're working with Eloquent.

See for further reading here on the Collection chunk() method here: https://laravel.com/docs/5.3/collections#method-chunk

Otherwise... Could you give some more context to what you're actually doing? It really sounds like you need pagination. What are you doing with the JSON data and how are you invoking the HTTP request, is it via ajax? Why do you need the whole 1000 records all at once?

If you really need the whole dataset of 1000 sent to the client, but 300 at a time then you don't want to use chunk. Think about what chunk() is for in the context of eloquent, it's not for getting chunks to return to the client a chunk at a time until it has the whole data set - it's for applying an action to a chunk then returning the whole set and the point of using it is so it doesn't take up too much memory at one time by loading the whole set to process the action on.

If you want the whole data set bit by bit and pagination won't work for your case (I'm yet to see why!) you will need to invoke the HTTP request several times to get the data bit by bit and specify in each request what you already have and what you need.

haakym
  • 12,050
  • 12
  • 70
  • 98
  • I have tried third answer, It is correct but my question is , If I want to show 1 thousand records, but it should send in the span of 300 records. Like first 300 and then 300 and then 300 , as json response. But not one thousand at once. – Aamir Oct 11 '16 at 09:30
  • Thanks alot..I got the concept now. :) – Aamir Oct 11 '16 at 10:04
  • 1
    @Aamir Awesome! Happy coding – haakym Oct 11 '16 at 10:06
  • 1
    @haakym Maybe you can help me. I want to implement chunk to my case. Look at this : https://stackoverflow.com/questions/51487769/how-to-insert-big-data-on-the-laravel – moses toh Jul 24 '18 at 03:11
  • chunk() method works both on Laravel 4 and 5. For Laravel 4, it gets an optional second parameter to preserve keys, which is default as false, and it's removed when I check 5.7 Collection reference. – Arda Feb 28 '19 at 13:23
9

Chunk is not for return data by request but for processing large tables on server - here is simple example showing how to use it (tested on Laravel 5.7):

User::chunk(100, function($users) {
    foreach ($users as $user) {
        // processing
    }
});
Kamil Kiełczewski
  • 85,173
  • 29
  • 368
  • 345
4

Instead of get() use chunk(). Eg. chunk(100, function($data) {})

ToNy
  • 71
  • 4
0
  1. can use the variable $counter and put
if ($counter>=100) { break; }
  1. can called on view with
use path/models/User;
Users->chunk(100, function($users) {
    foreach ($users as $user) {
        //
    }
});
Yun
  • 3,056
  • 6
  • 9
  • 28
rizky redjo
  • 75
  • 11
0

If you are using Laravel ORM Eloquent to load a large collection, you can use chunk() or cursor() methods as mentioned in the docs. This will lazy load the collection and reduce your memory usage.

An example:

Inspector::query()
        ->where('status', true)
        ->cursor() //for collection lazy loading
        ->each(function (Inspector $inspector) {
            $inspector->markAsInspected();
        });

Have a look at the doc links for more examples.

Abdelsalam Megahed
  • 1,281
  • 12
  • 13