2

In my script I wanted to clear the array elements to free memory from no longer used data.

I found myself in strange situation where using unset() causes:

( ! ) Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 16777224 bytes) in .../models/Persons.php on line 60

This is code part which causes this problem:

$chunks_count = count($this->xml_records_chunk['fnames']) - 1;
for ($num = 0; $num <= $chunks_count; $num++) {
      $chunks_count = count($this->xml_records_chunk['fnames']) - 1;

       $not_last = ($num < $chunks_count ? ',' : '');

       $new_records .= '(' . $this->xml_records_chunk['fnames'][$chunks_count] . ','
        . $this->xml_records_chunk['lnames'][$chunks_count] . ' , '
        . $this->xml_records_chunk['dobs'][$chunks_count] . ' , '
        . $this->xml_records_chunk['phones'][$chunks_count] . ' )' . $not_last;
       unset($this->xml_records_chunk['fnames'][$chunks_count]);
       unset($this->xml_records_chunk['lnames'][$chunks_count]);
       unset($this->xml_records_chunk['dobs'][$chunks_count]);
       unset($this->xml_records_chunk['phones'][$chunks_count]);
}

Script works just fine without unset.

Now the questions are:

  • Why unset causes memory exhaustion?
  • What is the correct way to unset unsused array elements in this case?

I've already checked this for example:

Ok null indeed works a bit other way since with it script dies on line 61 - 3rd unset.

Qirel
  • 25,449
  • 7
  • 45
  • 62
Volmarg Reiso
  • 437
  • 4
  • 11
  • Is `$this->xml_records_chunk['fnames']` an array or an object implementing array-like methods? – Blackhole Dec 15 '18 at 21:53
  • You might want to use [XMLReader](http://php.net/manual/en/book.xmlreader.php) or [XML Parser](http://php.net/manual/en/book.xml.php) to process XML without loading the entire file into memory. – Pinke Helga Dec 15 '18 at 22:23
  • I'm using XMLReader. `$this->xml_records_chunk` is an array. – Volmarg Reiso Dec 16 '18 at 11:49
  • I can't post solution as this was not solved however the problems I've encountered here were misunderstanding of how unset works along with garbage collection. Generally I think there might be some php bug. I had to deal with array of ~1milion elements and there was strange situation. I tried to controll memory usage (got lesson learned - not really worth it), so after 200.000 elements there was 70mb usage. But on 200.001 suddenly 140mb, and at this point memory usage was exceded yet still php moved forward toward the unset block. – Volmarg Reiso Dec 27 '18 at 14:37

1 Answers1

0

That is a good question why your unset breaks your memory. But you call $this->xml_records_chunk as variable in your code. So i would suggest that you have an array with all existing elements so you have allocated the complete memory already.

I think in that case you don't need to cleanup your array and memory because you have already allocated that memory. The GC is not that bad. So if you script doesn't use the variable anymore it's cleaned.

In your case i would suggest that you change your array structure and put the iterator to the first entry of your value something like this:

$this->xml_records_chunk[$chunks_count]['phones']

Then you have the following structure 

$this->xml_records_chunk[$chunks_count] = [
    'phones',
    '...',
    '...
]

Then you can clean with a single unset the complete array with

unset($this->xml_records_chunk[$chunks_count])

that could cause less problems and perhaps you could check the Iterator-Interface to iterate and delete your data.

René Höhle
  • 26,716
  • 22
  • 73
  • 82
  • I see Your point with just one unset but I think I will keep my structure, because I use it in many places. Well we all know that it's just small change but it can cause some additional bugs. Btw. when I was debugging current code, unset worked fine - I mean, array 0f ~240 000 elements shrunk to 239 999, so it works. Unset itself removes that variable. But I think I got an idea what might cause the problem at this point - will test that later, and will check iterator-interface as well. – Volmarg Reiso Dec 16 '18 at 11:48
  • Ok so I thought that assigning `$new_records` breaks the memory already, but it doesn't. Leaving just the unsets breaks memory on 2nd unset. Maybe You got idea if I can check with xdebug + phpStorm how much memory is used in given place? Or something like Htop for linux? – Volmarg Reiso Dec 16 '18 at 11:52
  • I've found this `xdebug_profiler` and `kcache` and I already see memory leaks - maybe there is the problem. – Volmarg Reiso Dec 16 '18 at 12:39