2

I'm generating about 700 PDFs with dompdf and saving them to the disk. Each one involves a database query to fetch the information for the PDF. When I run it I get about 2 minutes in and then it runs out of memory.

  1. Why am I running out of memory? Shouldn't each PDF generation consume x memory, then release it when finished?

  2. What's good practice in PHP for doing large operations like this to avoid strangling your server?

Some more info:

  • It generates many PDFs before running out of memory (some 100s)
  • The PDF it fails on is not abnormally large or special in any way. They're all between 4KB and ~500KB
  • Watching the memory usage as it processes it just slowly climbs in fits and starts until it runs out.

I can't get the wysiwyg to code-format properly, so here's a pastie for the snippets in question: http://pastie.org/3800751

Joren
  • 9,623
  • 19
  • 63
  • 104
  • 1
    Is it running out of memory after processing X number of PDFs, or before it finishes processing any? – Explosion Pills Apr 16 '12 at 21:42
  • Can you create the same PDF 700 times? Maybe it's just one too large PDF reaching the max. memory limit from the php.ini? – Smamatti Apr 16 '12 at 21:44
  • Seconded. We need much more than this to diagnose the problem. What happens if you just generate one PDF? Can you post pseudocode of your algorithm, or even better a minimal example of the problem? – jimw Apr 16 '12 at 21:44
  • 1
    Try to look into some kind of task queuing: [Run PHP Task Asynchronously](http://stackoverflow.com/questions/858883/run-php-task-asynchronously) – miku Apr 16 '12 at 21:44
  • Edited with more info. I wasn't sure what to include before. – Joren Apr 16 '12 at 21:56

4 Answers4

1

Your problem probably that you running your code asynchronously. Try running it synchronously, it might take a really long time but it will work. Also, make sure to dispose of your objects at the end of each loop.

Jason Foglia
  • 2,414
  • 3
  • 27
  • 48
1

Ideas:

  • Increase memory limit ini_set
  • Use a cron job, or generally try and run each pdf generation asynchronously with a queue.
  • Batch (split) processing into a number which can be processed within one page load / Redirect after each batch to same page using header / Keeping track of where you are on each script load with session / Beware of redirect limits on browsers
Ingmar Boddington
  • 3,440
  • 19
  • 38
  • My intention is to run it on a cron job. How do I run it in batches with a single cron execution? I've had no trouble with the redirect method you mention, but I'm not sure that works in a cron type setup? – Joren Apr 16 '12 at 23:39
  • You can chain together multiple executions in a single crontab command so they run sequentially I believe. – Ingmar Boddington Apr 17 '12 at 06:44
0

In general, large operations like this should be forked into child processes that each handle generating a single PDF. This should avoid the out of memory situation in case your PDF or DB libraries leak memory.

Clay Hinson
  • 932
  • 1
  • 6
  • 9
0

Try to change memory_limit at php.ini

Ahmet Can Güven
  • 5,392
  • 4
  • 38
  • 59