PHP How to Optimize foreach process for memory usage?

Question

I have some problem regarding memory issue using foreach on PHP. There is a process that using so much memory. The process is just take data from WebService and insert to database. But the data is big, almost thousand of data. It's really memory hungry, almost eat 2gigs of ram. Is there anyway to optimize my code ? I have tried unset() and = NULL but doesn't free up the memory. Here my code, i am using codeigniter :

$proxy = $this->proxy;
$token = $this->token;
$table = 'nilai';

$qk =$this->db->query("SELECT id_reg_pd from $databaseakm");
$tampan = $qk->result();


foreach($tampan as $keyz0 => $data0){


    $id_reg_pd = $data0->id_reg_pd;
        $filter3 = "p.id_reg_pd='$id_reg_pd'";
        $order3= "";
        $limit3 = 500;
        $offset3 = 0;
        $tampan3 = $proxy->GetRecordset($token, $table, $filter3, $order3, $limit3, $offset3);


$_datas = array_chunk($tampan3["result"], 50);      

foreach($_datas as $key => $data)

    {
        $this->db->trans_start();
        $this->db->query("UPDATE temp_tabel SET status_isi = 2 WHERE nama_tabel = '$databasenilai'");
        $this->db->insert_batch($databasenilai, $data);
        $this->db->trans_complete();
        $tampan3 = NULL;
        $data = NULL;
        unset($tampan3);
        unset($data);
    }   
$tampan3 = NULL;
        $data = NULL;
        unset($tampan3);
        unset($data);
}
$tampan3 = NULL;
        $data = NULL;
        unset($tampan3);
        unset($data);

you could create a master script that call a child script multiple times using an `offset` as a parameter and the child script uses the `offset` in the main query as such: `SELECT id_reg_pd from $databaseakm LIMIT $offset, 100` — Constantin Galbenu, Mar 22 '17 at 19:03
@Jay Blanchard, sorry i am confusing the english, it's thousands I mean. — Billy Adelphia, Mar 22 '17 at 19:12
[Prepared statements](http://stackoverflow.com/a/23033730/1628790) can optimize your `update`. [Generators](http://php.net/manual/en/language.generators.overview.php) will also reduce a lot of memory usage if you retrieve the result set using `yield`. — Gabriel Heming, Mar 22 '17 at 19:14
More about generators: https://evertpot.com/switching-to-generators/ — Gabriel Heming, Mar 22 '17 at 19:24
@GabrielHeming I can't find the prepared statement on codeigniter, and how to implement yield on my code ? I never use it before — Billy Adelphia, Mar 22 '17 at 19:25
About generators, there's the link above. If you change the return of the first query, you can retrieve only a row at a time. Then, there's the second question. For the `prepared statements`, don't get as a personal offense, but code igniter sucks (I developed with him for about a year and I'm here to help you). Take a look here: http://stackoverflow.com/a/14156676/1628790 — Gabriel Heming, Mar 22 '17 at 19:30

score 1 · Answer 1 · answered Feb 26 '18 at 17:14

1

Return an object and use while loop instead of for each. For each tries to load the entire data into memory and loop over it, while loop will do one at a time and go get the next one

answered Feb 26 '18 at 17:14

Shinningweb

11
1

PHP How to Optimize foreach process for memory usage?

1 Answers1