4

I have a resource intensive script that takes about 30 minutes to complete. As its running it gradually consumes RAM. Once its finished it has consumed about 300mb. I need to run this script multiple times so this is becomming an issue. I have to restart the server every time i want to start running the script again as ram usage hits 100% and the script stalls.

How can i force php or mysql to release the memory? PHP Version 5.4.29 Centos VPS 2GB ram Max memory for a php scripts is set to 512mb.

The script makes a large amount of SQL queries. Should be in the 1000s. It also makes calls to 5 different APIs 50 times. It selects 50 row from a db table and loops through it, making calls to APIs etc to update it.

Garbage collection is enabled var_dump(gc_enabled()); returned true

             total       used       free     shared    buffers     cached
Mem:          2048       1522        525          0          0        182
-/+ buffers/cache:       1339        708
Swap:          128          0        128
Dan Hastings
  • 3,241
  • 7
  • 34
  • 71
  • 2
    are you running the php script via the shell? if so check what processes are left after your script ends. PHP is not a continually running process so should not be found in memory after the script ends. – Ryan Vincent Nov 29 '14 at 20:27
  • i was initially running it via wget, but now im just running it through the browser. it isnt in memory after the script ends. mysql is showing up as using 4% ram and thats all i can see using memory. there doesnt seem to be any process using the ram. its just missing – Dan Hastings Nov 29 '14 at 20:45
  • Define "just missing". What makes you think the memory is not being released? (You may want to refer to http://www.linuxatemyram.com/) –  Nov 29 '14 at 20:46
  • i have posted the result of free -m. I have used the top command to see what processes are using the ram and nothing is using it. httpd is using 1% and mysql using 4. there are a bunch of other processes using less than 1% – Dan Hastings Nov 29 '14 at 20:49
  • It usually helps to see the code. You're not cleaning up after doing work, or during work. You should do that. – N.B. Nov 29 '14 at 21:22
  • 1
    Could you show the results of `free -m` before running the script at all, and then after running it once, twice, three times, etc until we can see the pattern? Eventually when you run it the machine runs out of RAM? – Erick Robertson Nov 30 '14 at 14:47
  • 1
    Actually, to one-up that, how about the diff of top before and after the script runs? Please put special emphasis on the `mysqld` line! – Erick Robertson Dec 01 '14 at 04:30
  • possible duplicate of [Is the memory allocated by PHP in a single request always released at the end?](http://stackoverflow.com/questions/7456333/is-the-memory-allocated-by-php-in-a-single-request-always-released-at-the-end) – RandomSeed Mar 23 '15 at 00:27

4 Answers4

2

When script is completed all resources should be freed, so you should not be in need of doing anything extra unless you use PHP/lib that seems to be buggy and leaks memory

EDIT

Memory leak is result of a bug and in languages like PHP if memory leak happens in PHP or its modules you will not be usually able to fix it other way by updating PHP or used modules to never version and hope this will come with leak fixed. You may try to narrow down what component is leaking and then report to its authors. As temporary solution you can work this issue around by restarting your httpd periodically which shall free all memory allocated by php or modules. But that's quick workaround only

Marcin Orlowski
  • 72,056
  • 11
  • 123
  • 141
  • How can i get to the bottom of a leak? or more to the point can i add code to my script that will allow me to clear memory that has leaked? i used the top command and there is nothing using this ram. Php is urrently using 200mb ram so its not php either. – Dan Hastings Nov 29 '14 at 21:01
0

You can set variable to null to free memory immediately.

I think you need to set up the database memory limit by adjusting the buffer size

How to set memory limit in my.cnf file

Community
  • 1
  • 1
TYPCN
  • 11
  • 1
  • 2
0

This is not PHP which is taking up this RAM.

Once the php process is complete, it returns all of its RAM to the system. So this is not PHP which is causing this.

Some other application on the server is taking up additional memory, causing less to be available to php when it runs. From what you've described, it's easy to imagine that it's MySQL. You said you're running 1000's of queries. MySQL does profiling and caching internally to optimize those queries. This would absolutely cause it to take up additional memory after you've run the PHP script.

Also check on the other services. If any of the services are being called from your script, yet continue to run between script executions, they could also be taking a larger footprint in memory. Any of these services could be configured for smaller memory footprints, although perhaps some cannot.

Certainly, though, the solution is running PHP and MySQL on two different servers. It is far simpler to configure each application to use its entire server than to configure them to share one server nicely. 2 GB of RAM seems low for a script which requires 300mb to run.

If your server resources are fixed, it's possible you can modify the script to run longer but take less memory, too. Maybe you can load the largest data in pieces and flush each result out before loading the next. But to me, this would only be a temporary solution.

Erick Robertson
  • 32,125
  • 13
  • 69
  • 98
-1

Check out the ob_implicit_flush() function.

PHP Docs

ct12
  • 27
  • 1
  • 5