memory_get_peak_usage() with "real usage"

Question

If the real_usage argument is set to true the PHP DOCS say it will get the real size of memory allocated from system. If it's false it will get the memory reported by emalloc()

Which one of these 2 options returns the max. memory allocated relative to the memory limit value in php.ini ?

I want to know how close was the script to hit that limit.

I would like to point you to a presentation by Julien Pauli http://www.youtube.com/watch?v=sm1HUrnsxLI for the php uk conference 2013, where he talks about how memory works inside PHP. — mpratt, Apr 30 '13 at 20:13

score 155 · Accepted Answer · edited May 23 '17 at 12:02

155

Ok, lets test this using a simple script:

ini_set('memory_limit', '1M');
$x = '';
while(true) {
  echo "not real: ".(memory_get_peak_usage(false)/1024/1024)." MiB\n";
  echo "real: ".(memory_get_peak_usage(true)/1024/1024)." MiB\n\n";
  $x .= str_repeat(' ', 1024*25); //store 25kb more to string
}

Output:

not real: 0.73469543457031 MiB
real: 0.75 MiB

not real: 0.75910949707031 MiB
real: 1 MiB

...

not real: 0.95442199707031 MiB
real: 1 MiB

not real: 0.97883605957031 MiB
real: 1 MiB

PHP Fatal error:  Allowed memory size of 1048576 bytes exhausted (tried to allocate 793601 bytes) in /home/niko/test.php on line 7

Seems like real usage is the memory allocated from the system - which seems to get allocated in larger buckets than currently needed by the script. (I guess for performance reasons). This is also the memory the php process uses.

The $real_usage = false usage is the memory usage you actually used in your script, not the actual amount of memory allocated by Zend's memory manager.

Read this question for more information.

In short: to get how close are you to the memory limit, use $real_usage = true

edited May 23 '17 at 12:02

Community

1
1

answered Apr 26 '13 at 14:53

Niko Sams

4,304
3
25
44

8

The Zend engine allocates memory in 256K chunks. The "real usage" value is the sum of all these chunks. That's actually the value used to trigger the memory exhaustion error: `if (segment_size < true_size || heap->real_size + segment_size > heap->limit) { /* Memory limit overflow */`. – cleong May 01 '13 at 13:18
2

The "not real" value is the sum of the number of bytes requested by calls to `emalloc` (plus bytes for headers and memory alignment). It doesn't reflect memory wasted due to blocks not fitting into space remaining in already allocated segments. If you change your example to allocate (1024 * 256) bytes and a 2M limit, the difference of two will become more apparent. – cleong May 01 '13 at 13:31
@Niko, Why did you use memory_get_peak_usage instead of memory_get_usage ? Shouldn't we gc_disable() and use memory_get_usage to get a more accurate result? – Pacerier Jul 13 '13 at 07:34
@Pacerier the question was to get how close to the limit the script is - for that peak makes sense I'd say – Niko Sams Jul 13 '13 at 12:59
I think it's better to write if(ini_set('memory_limit', '1M')) for those people who love copy paste and already set the memory_limit to -1 and ini_set doesn't work work them just in case :) – h0mayun Mar 02 '14 at 09:42
6

As @cleong has explained, this answer is actually wrong, despite all the upvotes. The `memory_get_usage(true)` returns value that should be compared to `memory_limit`. The example given in the answer is way too simple as there's no "wasted" memory". What happens is that the "real" allocated memory needs to be increased from "1 MiB" to "1.25 MiB" and that's what triggers the fatal error. I have a complex batch script with memory limit of 120 MiB that have "not real" allocated memory of only "80 MiB" when it is aborted because the "real" allocated memory reaches the limit. – Martin Prikryl May 13 '14 at 07:44

score 43 · Answer 2 · edited Feb 07 '22 at 20:31

Introduction

You should use memory_get_usage(false) because what you want is memory used not memory allocated.

What's the Difference

Your Google Mail might have allocated 25MB of storage for you but it does not mean that is what you have used at the moment.

This is exactly what the PHP doc was saying

Set this to TRUE to get the real size of memory allocated from system. If not set or FALSE only the memory used by emalloc() is reported.

Both arguments would return memory allocated relative to the memory limit but the main difference is:

memory_get_usage(false) give the memory used by emalloc() while memory_get_usage(true) returns milestone which can be demonstration here Memory Mile Store

I want to know how close was the script to hit that limit.

That would take some maths and might only work in loops or specific use cases. Why do I say that?

Imagine

ini_set('memory_limit', '1M');
$data = str_repeat(' ', 1024 * 1024);

The above script would fail before you even get the chance to start start checking memory.

As far as I know the only way I can check memory used for a variable or specific section of PHP is:

$start_memory = memory_get_usage();
$foo = "Some variable";
echo memory_get_usage() - $start_memory;

See Explanation, but if you are in a loop or recursive function you can use maximum memory usage to estimate safely when memory peak would be reached.

Example

ini_set('memory_limit', '1M');

$memoryAvailable = filter_var(ini_get("memory_limit"), FILTER_SANITIZE_NUMBER_INT);
$memoryAvailable = $memoryAvailable * 1024 * 1024;

$peakPoint = 90; // 90%

$memoryStart = memory_get_peak_usage(false);
$memoryDiff = 0;

// Some stats
$stat = array(
        "HIGHEST_MEMORY" => 0,
        "HIGHEST_DIFF" => 0,
        "PERCENTAGE_BREAK" => 0,
        "AVERAGE" => array(),
        "LOOPS" => 0
);

$data = "";
$i = 0;
while ( true ) {
    $i ++;
    
    // Get used memory
    $memoryUsed = memory_get_peak_usage(false);
    
    // Get Difference
    $memoryDiff = $memoryUsed - $memoryStart;
    
    // Start memory Usage again
    $memoryStart = memory_get_peak_usage(false);
    
    // Gather some stats
    $stat['HIGHEST_MEMORY'] = $memoryUsed > $stat['HIGHEST_MEMORY'] ? $memoryUsed : $stat['HIGHEST_MEMORY'];
    $stat['HIGHEST_DIFF'] = $memoryDiff > $stat['HIGHEST_DIFF'] ? $memoryDiff : $stat['HIGHEST_DIFF'];
    $stat['AVERAGE'][] = $memoryDiff;
    $stat['LOOPS'] ++;
    $percentage = (($memoryUsed + $stat['HIGHEST_DIFF']) / $memoryAvailable) * 100;
    
    // var_dump($percentage, $memoryDiff);
    
    // Stop your script
    if ($percentage > $peakPoint) {
        
        print(sprintf("Stoped at: %0.2f", $percentage) . "%\n");
        $stat['AVERAGE'] = array_sum($stat['AVERAGE']) / count($stat['AVERAGE']);
        $stat = array_map(function ($v) {
            return sprintf("%0.2f", $v / (1024 * 1024));
        }, $stat);
        $stat['LOOPS'] = $i;
        $stat['PERCENTAGE_BREAK'] = sprintf("%0.2f", $percentage) . "%";
        echo json_encode($stat, 128);
        break;
    }
    
    $data .= str_repeat(' ', 1024 * 25); // 1kb every time
}

Output

Stoped at: 95.86%
{
    "HIGHEST_MEMORY": "0.71",
    "HIGHEST_DIFF": "0.24",
    "PERCENTAGE_BREAK": "95.86%",
    "AVERAGE": "0.04",
    "LOOPS": 11
}

Live Demo

This may still fail

It may fail because after if ($percentage > $peakPoint) { this still still add to do additional task with also consumes memory

        print(sprintf("Stoped at: %0.2f", $percentage) . "%\n");
        $stat['AVERAGE'] = array_sum($stat['AVERAGE']) / count($stat['AVERAGE']);
        $stat = array_map(function ($v) {
            return sprintf("%0.2f", $v / (1024 * 1024));
        }, $stat);
        $stat['LOOPS'] = $i;
        $stat['PERCENTAGE_BREAK'] = sprintf("%0.2f", $percentage) . "%";
        echo json_encode($stat, 128);
        break;

If the memory to process this request is greater than the memory available the script would fail.

Conclusion

It's not a perfect solution, but check for memory at intervals and if it exceeds the peak (eg 90%), exit instantly and leave the fancy stuff

what if i have two script in parallel or many request , di the memory_get_usage() function return the memory used for all those script executed in the same time or only the actual script ? — Yassine CHABLI, Apr 11 '19 at 10:19

Glitch Desire · Answer 3 · 2013-09-19T19:05:47.060

7

real_usage false reports the usage your script used. This will be the more accurate of the two.

real_usage true reports the memory allocated to your script. This will be the higher of the two.

I'd probably use true if I was trying to compare, as your script would never be allocated more than memory limit, and would continue to run as long as it (plus all other scripts) didn't exceed that usage.

edited Sep 19 '13 at 19:05

answered Apr 26 '13 at 13:57

Glitch Desire

14,632
7
43
55

1

It's exactly the opposite: `false` is the memory the script *used*, `true` is the memory *allocated*. – BenMorel Sep 19 '13 at 16:00
1

@Benjamin Yeah, not sure why I got that so blindly wrong. Umm, fixed. – Glitch Desire Sep 19 '13 at 19:06

score 2 · Answer 4 · edited Jun 20 '20 at 09:12

as per PHP memory_get_usage

real_usage

Set this to TRUE to get total memory allocated from system, including unused pages. If not set or FALSE only the used memory is reported.

so to get the memory used by your script you should use memory_get_usage() as default real_usage is false.

if you want to get the memory allocated by the system but don't care how much was actually used, use memory_get_usage(true);

score -1 · Answer 5 · answered Apr 15 '19 at 11:01

-1

<!-- Print CPU memory and load -->
<?php
$output = shell_exec('free');
$data = substr($output,111,19);
echo $data;
echo file_get_contents('/proc/loadavg');
$load = sys_getloadavg();
$res = implode("",$load);
echo $res;
?>

answered Apr 15 '19 at 11:01

Rahul Rohewal

9
2

2

Welcome to Stackoverflow! could you please tell us what the answer is? Not just the code but also how you figured out how to fix the question. Thanks! – Gilles Heinesch Apr 15 '19 at 11:22
While your answer might provide some potentially useful information, it is not relevant to the question being asked. You might want to explain how your answer relates to the question being asked. – Moritur Mar 18 '20 at 15:02

memory_get_peak_usage() with "real usage"

5 Answers5

Linked

Related