PHP script memory limit and memory optimimization

Question

I have a php script to scrap a website (text files only). After running for few hours I noticed the script to stop for reaching the memory limit. I know I can increase the limit, but since the files the script loads are onlty HTML files I explain the reaching of the limit only with the inability of the script to empty the memory after each loop. Could I optimize my script's memory management by flush()ing its memory regularly?

If you show the code, perhaps someone could spot where memory isn't free'd. `flush()` is for dealing with output buffers. — alex, Mar 23 '13 at 11:54
Note the PHP version number as well. Memory management is quite different in PHP 5.2 and earlier. — cleong, Mar 23 '13 at 11:55
My question was not related to my particular script. I just want to understand conceptually when memory need to be freed explicitly in the code. — CptNemo, Mar 23 '13 at 12:00
@CptNemo You shouldn't ever need to manually manage memory in PHP, but you do need to write your code in ways that help the memory manager do its job. So it depends entirely on the kind of structures a particular script uses. — IMSoP, Mar 23 '13 at 12:04

score 4 · Answer 1 · answered Mar 23 '13 at 13:26

In general, you shouldn't need to manually manage memory in PHP, as it has a high-level Memory Manager built in to the Zend Engine which takes care of this for you. However, it is useful to know a bit about how this works in order to better understand why your code is running out of memory.

As a very basic overview, PHP frees memory based on a "refcount" of how many variables are referencing a particular piece of data. So if you say $a = 'hello'; $b = $a;, a single piece of memory containing the string 'hello' will have a refcount of 2. If you call unset() on either variable, or they fall out of scope (e.g. at the end of the function they were defined in), the refcount will decrease. Once the refcount reaches zero, the data will be deleted and the memory freed. Note that "freed" in this case means freed for use by other parts of that PHP script, not necessarily freed back to the Operating System for use by other processes.

There are a few differences between PHP versions worth knowing:

The reference counting mechanism described above doesn't work if you have circular references (e.g. $obj1->foo = $obj2; $obj2->bar = $obj1;) because the reference count never reaches zero. In PHP 5.2 and earlier, this meant that such circular references led to memory leaks, and had to be manually handled by the programmer. In PHP 5.3, a "Garbage Collector" was added specifically to handle this case. It does not replace the normal refcount mechanism, but if circular references are common in your code, it may be worth reading up on.
PHP 5.4 included a large number of optimizations to the way PHP allocates and uses memory. AFAIK, none of these change the fundamental recommendations of how to write efficient code, they are just a good reason to upgrade your PHP version if you can.

Other than that, there are a few common tips for writing PHP code that makes good use of memory:

Make sure unused variables are discarded when no longer needed. In a well-structured program, this is often a non-issue, because most variables will be local to a particular function; when the function exits, they will go out of scope, and be freed. But if you are creating large intermediate variables, or dynamically creating large numbers of variables, manually calling unset() may be a good idea. And if your code is very linear, or uses large numbers of global and static variables, just refactoring it into a more modular structure may improve its memory performance as well as its readability, maintainability, etc.
Assigning or passing a variable by reference ($foo = &$bar) may cause PHP to use more memory than a straight assignment ($foo = $bar). This is because PHP uses a "Copy On Write" mechanism to to store variables with the same content in one location of memory, but reference assignment conflicts with this mechanism, so PHP has to copy the variable early.
Objects are more memory-hungry than scalar values (int, boolean, string) or arrays. This is one of the things that has been much improved in PHP 5.4, but is still worth thinking about - although obviously not to the exclusion of writing well-structured code!

score 1 · Answer 2 · edited May 23 '17 at 12:10

1

You can unset variables as you no longer need them (e.g. unset($var) or $var = null). If you're on PHP 5.3 or later, you can also explicitly call the garbage collector: see gc_collect_cycles() and gc_enable().

Some functions seem to be worse than others. I recently found that array_merge_recursive() did horrible things to my code's memory footprint.

If you want to be able to analyse where the memory's going, you can use tools like Xdebug or XHProf/XHGui to help. e.g. Xdebug and tracing memory usage and Profiling with XHProf

PHP script memory limit and memory optimimization

2 Answers2