1

I am creating a lot of child processes using Parallel::ForkManager and then killing them using kill 'KILL', $_ for $pm->running_procs, but there is a big memory leak in doing this:

Note that using kill('KILL', ...) on a pseudo-process() may typically cause memory leaks, because the thread that implements the pseudo-process does not get a chance to clean up its resources.

Source: http://perldoc.perl.org/perlfork.html#Behavior-of-other-Perl-features-in-forked-pseudo-processes

How can I kill the child processes and not cause the big memory leak?

CJ7
  • 22,579
  • 65
  • 193
  • 321
  • Do you have to kill them by hand? The last time I looked, it turned out that there may not be a way to 'gently' kill a process on windows. See [this post](http://stackoverflow.com/questions/37757264/how-can-i-gently-kill-a-process-in-activestate-perl/37757471#37757471) for practically the same issue. – zdim Nov 22 '16 at 05:49
  • I meant to say, the best way appears to be `system("taskkill /t /pid $pid");`. See the linked post. – zdim Nov 22 '16 at 05:56
  • @zdim This is on a Unix based system – CJ7 Nov 22 '16 at 06:41
  • 2
    Oh ... the doc link is about emulation of fork on Windows, and the "_pseudo-process_" phrase is about that, too. If you are on UNIX you don't want to (ever) send the `SIGKILL` signal (`KILL` in Perl), for that precise reason. Send the `SIGTERM` (`TERM` in Perl), that allows the process to do (some) cleanup. Then, in those child processes, you write a signal handler for `SIGTERM` where you can shut down as appropriate. If those are external commands you hope that they clean up on `SIGTERM`. – zdim Nov 22 '16 at 06:46
  • Ok, all I want to do is free up the memory. Do I need a signal handler for that? – CJ7 Nov 22 '16 at 06:49
  • Well, yes -- if a `SIGTERM` signal is not handled its _disposition_ (default action) is to terminate the process, right there. If you _are_ ending up with leaks then the child has to catch it, and in the handler you can write code that will do what is needed to free memory. So when the child gets hit by the signal the code in the handler runs. Then, for `SIGTERM` the process must exit. (Other than that and `SIGKILL` all others can be survived.) This is covered in [perlipc](http://perldoc.perl.org/perlipc.html) (and in bunch of other docs). – zdim Nov 22 '16 at 06:51
  • Can the handler just call `exit` or `die`? Shouldn't that free up the memory? – CJ7 Nov 22 '16 at 06:54
  • Right, I think that it should, since (I think) it allows `END` phase and the global destruction phase to run (I am just not sure for `SIGTERM`). I just don't know how those processes leak memory. But you are right, the first thing to try would be to just throw `die` in the handler. – zdim Nov 22 '16 at 06:56
  • 1
    Thinking about it some more, I don't know how Perl (interpreter) handles `SIGTERM` -- try first without a handler, maybe it runs the global destruction anyway. If it doesn't then add a handler with `die`. If _that_ doesn't work then you may have to do more. – zdim Nov 22 '16 at 07:02
  • I confirmed that with with unhandled `SIGTERM` things never get to `END` phase, so one does need a handler. However, with a mere handler in place, just printing to screen (no die), the `END` blocks run etc. – zdim Nov 22 '16 at 18:56
  • I just re-read my comments and see that I made an incorrect implication: it is `SIGSTOP`, along with `SIGKILL`, that cannot be caught. The `SIGTERM` can, and the program then need not terminate. However, normally it's a good idea to respect `SIGTERM` and exit, after doing whatever is needed for a clean exit. – zdim Nov 22 '16 at 22:10
  • Ok, in the handler I would like to just free up the memory and not print anything to the screen. Is this possible? – CJ7 Nov 22 '16 at 22:40
  • To exit gracefully you need `die` -- without it the program continues, since the signal got disabled by having a handler. The `die` on its own prints a message, what can be suppressed by adding a newline to it. So `$SIG{TERM} = sub { die "\n" };` – zdim Nov 22 '16 at 22:54
  • You prefer `die` over `exit`? With `exit` you don't have to worry about the newline. – CJ7 Nov 23 '16 at 01:17
  • Good point -- no, I don't. It's just that it thought it was a touch more flexible in general. The `exit` is just as fine (or better), and calls `END` blocks and goes through destruction. – zdim Nov 23 '16 at 06:37

0 Answers0