I've been looking around but haven't been successful and finding a solution for my case.
I noticed that when running a PHP on my EC2 instance it would require a lot of memory (This at the same time seems to buff the memory needed by the HTTPD process as the HTTPD process memory will go down if I disable the PHP extension)
To perform a simple check, I have Apache running (HTTPD) and a very simple php program
<?php
while(true) {
sleep(1);
$x = 1;
}
Then, on a separate console, I run the top
command and SHIFT+m
to sort by memory
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
31155 apache 20 0 936m 299m 27m S 0.0 7.6 0:00.48 httpd
31141 apache 20 0 935m 298m 27m S 0.0 7.5 0:00.41 httpd
31147 apache 20 0 936m 297m 26m S 0.0 7.5 0:00.41 httpd
31121 root 20 0 839m 294m 29m S 0.0 7.5 0:00.93 httpd
31505 ec2-user 20 0 630m 279m 16m S 0.0 7.1 0:00.81 php
31264 apache 20 0 847m 271m 6404 S 0.0 6.9 0:00.18 httpd
31266 apache 20 0 847m 271m 6404 S 0.0 6.9 0:00.18 httpd
31123 apache 20 0 1039m 271m 6400 S 0.0 6.9 0:00.30 httpd
31143 apache 20 0 847m 271m 6400 S 0.0 6.9 0:00.31 httpd
31145 apache 20 0 847m 271m 6400 S 0.0 6.9 0:00.30 httpd
31149 apache 20 0 847m 271m 6400 S 0.0 6.9 0:00.31 httpd
31151 apache 20 0 847m 271m 6400 S 0.0 6.9 0:00.31 httpd
31157 apache 20 0 847m 271m 6400 S 0.0 6.9 0:00.30 httpd
31261 apache 20 0 847m 271m 6244 S 0.0 6.9 0:00.17 httpd
You'll notice that despite the Apache service just being started and being inactive, and a simple php script, they are all consuming a big amount of memory.
From a similar question How to optimize the php process memory usage?
I went and ran pmap -d 31505
to which I found two main entries
31505: php /home/ec2-user/infinite.php
Address Kbytes Mode Offset Device Mapping
0000000000400000 3044 r-x-- 0000000000000000 0ca:00001 php-5.5
...
00000000027a3000 265876 rw--- 0000000000000000 000:00000 [ anon ]
00007f9f1e249000 1028 rw--- 0000000000000000 000:00000 [ anon ]
...
00007f9f2de29000 32 rw--- 0000000000000000 000:00000 [ anon ]
00007f9f2de31000 103588 r---- 0000000000000000 0ca:00001 locale-archive
00007f9f3435a000 120 r-x-- 0000000000000000 0ca:00001 libselinux.so.1
00007f9f34378000 2044 ----- 000000000001e000 0ca:00001 libselinux.so.1
...
From this, I was able to notice that 265Mb were used by some [ anon ]
and 103Mb by a file locale-archive
Doing some more research, I understood a bit more about locale-archive
and I found the file on the system /usr/lib/local/locale-archive
which file is about 103Mb which matches.
I found this other link which I tried to follow to reduce the size of the locale-archive
file, but I ran into all the problems noted on the same link and wasn't able to resize it.
(Problem: /usr/sbin/build-locale-archive: cannot read archive header)
https://unix.stackexchange.com/questions/90006/how-do-i-reduce-the-size-of-locale-archive
So, it seems that somewhere in between this [ anon ]
and the size of locale-archive
is what causes the PHP process to consume a lot of memory.
The Questions
- What is
[ anon ]
and how to reduce it? - Would reducing the size of
locale-archive
help?
If so, how can I reinstallglibc-common
with only a single locale likeen
?
The Reason of this need
I need to run a PHP script in parallel with a minimum of 10 processes.
While trying, I noticed that many processes were failing to start due to memory allocation insufficiency.
While trying to figure out this, is when I noticed that httdd
and php
processes were just consuming an unreal amount of memory
Executing free -m
yields
total used free shared buffers cached
Mem: 15042 14558 484 26 216 695
-/+ buffers/cache: 13645 1397
Swap: 0 0 0
So, I virtually only have about 400Mb to run these scripts, which only allow me to run at most 2 based on the default 294Mb used by a single php script that only uses about 1kb of memory, but the rest is the PHP process on itself.
Now, though httpd
is consuming most of the memory, it seems that it is an indirect result of the underlying PHP
module launched with the service. (Compared to a test where I disabled the PHP module and the httpd
processes went down to a 1/3 of the memory they were using)
So, it definitely looks that the PHP
process is the one to blame, which is affected by whatever [ anon ]
is, and indirectly by the size of locale-archive
If I can get the PHP process to not use so much memory, not only will I be able to run the 10 scripts (I could run 20 or 30) without problem and Apache would even have more memory to serve even more connections I think.