How to avoid processes?

Question

I do ZFS remote replication from a master host to a slave host, where I have a Perl script that runs on the master host.

For each filesystem it ssh to the remote host and start mbuffer in listening mode and then the script continues and then send the data. On success mbuffer should exit by itself.

The problem

It was quite difficult to start mbuffer on the remote host over ssh and then be able to continue in the script. I ended up doing what you can see below.

The problem is that until the script exits it leaves behind <defunct> processes one for each filesystem.

Question

It is possible to avoid having the <defunct> processes?

sub mbuffer {
    my ($id, $zfsPath) = @_;

    my $m = join(' ', $mbuffer, '-I', $::c{port});
    my $z = join(' ', $zfs, 'receive', , $zfsPath);
    my $c = shellQuote($ssh, $::c{slaves}{$id}, join('|', $m, $z));

    my $pm = Parallel::ForkManager->new(1);
    my $pid = $pm->start;
    if (!$pid) {
        no warnings;  # fixes "exec" not working
        exec($c);
        $pm->finish;
    }

    sleep 3; # wait for mbuffer to listen

    return $pid;
}

A parent process must always call `wait` (or one of its variants) on their child processes in order to let the kernel know that the terminated child can be cleaned up. [This question](https://stackoverflow.com/questions/9164316/c-fork-without-wait-defuncts-execl) has some answers that might point you in the right direction. — Thomas, Aug 09 '17 at 13:29
Quickest fix is to set `$SIG{CHLD}='IGNORE'`. See [`perldoc -f fork`](http://metacpan.org/pod/perlfunc#fork) — mob, Aug 09 '17 at 13:41
A defunct process or a zombie process is a process that terminated without it's parent calling `wait` on it. As such the kernel keeps the entries of the terminated child process, so when the parent calls `wait` it returns the needed information. To avoid zombies, the parent process needs to wait on its children. — GIZ, Aug 09 '17 at 13:50

ikegami · Accepted Answer · 2020-09-14T16:46:12.343

When you create a process, it sticks around until its parent reaps it. (If its parent exits first, it will get auto-reaped.) A process can reap its children using wait or waitpid. It can also cause its children to be automatically reaped by using local $SIG{CHLD} = 'IGNORE'; before creating the child.

Note that Parallel::ForkManager is not the right tool for the job of launching a single child. It's not its purpose to spawn a single worker.

use String::ShellQuote qw( shell_quote );

sub mbuffer {
    my ($id, $zfsPath) = @_;

    my $mbuffer_cmd = shell_quote($mbuffer, '-I', $::c{port});
    my $zfs_cmd     = shell_quote($zfs, 'receive', $zfsPath);
    my $remote_cmd  = "$mbuffer_cmd | $zfs_cmd";
    my $local_cmd   = shell_quote($ssh, $::c{slaves}{$id}, $remote_cmd);

    # open3 will close this handle.
    # open3 doesn't deal well with lexical handles.
    open(local *CHILD_STDIN, '<', '/dev/null') or die $!;    

    return open3('<&CHILD_STDIN', '>&STDOUT', '>&STDERR', $local_cmd);
}

IPC::Open3 is quite low level, but it's the closest to your existing code. Better ways of launching processes include IPC::Run3 and IPC::Run.

zdim · Answer 2 · 2017-08-22T08:30:51.300

1

For one, there is no reason to use P::FM with one process. Also, it is disadvantagous here since you give up finer control over process management.

But the direct error here is in the use of exec; this post addresses only that.

The exec call replaces what is in the process with another program and never returns. So code in the child that is after exec doesn't run (see docs). Thus $pm->finish is left hanging, the child process never gets reaped, and the OS keeps its info in the process table so there's a defunct/zombie.

Here is a basic way to fire off another program using exec directly

my $cmd = '...';

my $pid = fork  // die "Can't fork: $!";

if ($pid == 0) {
    exec $cmd;
    die "exec shouldn't return: $!";
}
my $gone = waitpid $pid, 0;

if    ($gone > 0) { say "Child $gone exited with $?" }
elsif ($gone < 0) { say "No $pid process ($gone), reaped already?" }
else              { say "Process $pid still running?" }

Here the child inherits standard streams from the parent. Also, error reporting is coarse (imprecise) in some cases, see comments thanks to ikegami.

A more detailed and faithful replacement for what you do is in ikegami's answer.

edited Aug 22 '17 at 08:30

answered Aug 09 '17 at 17:25

zdim

64,580
5
52
81

That makes it look like the child was successfully launched when `exec` fails. That's why I recommend `open3` instead (or something even better). – ikegami Aug 09 '17 at 18:23
@ikegami This is what I don't get -- that way I see it is that if `fork` fails there's a message for that, while if I get the `die` then the child _was_ created successfully. Or do you mean that `waitpid` won't reveal where the problem was? I suspect I may be missing stuff here. (Btw, I didn't mean to say that this is a complete and robust way to go, I hope the post's clear on that.) Thank you for your comments. – zdim Aug 09 '17 at 19:34
Your way can't distinguish between a failure to launch `ssh` and `ssh` doing `exit(2)`. That's not how Perl's `system` works. That's not C's `system` works. That's not how `bash` works. In all of those cases, `exec` failures and errors preparing to `exec` are made to appear to be failures in launching the program, not errors being returned by the launched program. (For Perl's `system`, it'll return `$? = -1` and sets `$!`.) – ikegami Aug 09 '17 at 19:51
@ikegami OK, I see what you mean. I am _loosely_ aware of coarseness (imprecision) of error reporting here, thank you for this. Adding a basic comment now, will do more (or remove) as soon as I get to it. Thank you again. – zdim Aug 09 '17 at 22:09
I'm not saying you need to change anything; I'm just pointing out a limitation. But it is one that was addressed by the solution posted 25 minutes before yours. – ikegami Aug 09 '17 at 23:00
@ikegami Thank you for saying it, and for explaining it. I think that I need to edit, to make the distinction clear. I posted only to provide a really simple example, mostly in order to point out OP's misuse of `exec`. I meant to keep it simple, and didn't fully appreciate the deficiency in error reporting and didn't know that modules do that better (I deferred to the other answer in what I thought mattered). But I need to play with such errors using `Run` modules and `open3` first, to see how they do. – zdim Aug 09 '17 at 23:24

How to avoid processes?

2 Answers2