21

I want to spawn long-running child processes that survive when the main process restarts/dies. This works fine when running from the terminal:

$ cat exectest.go
package main

import (
        "log"
        "os"
        "os/exec"
        "syscall"
        "time"
)

func main() {
        if len(os.Args) == 2 && os.Args[1] == "child" {
                for {   
                        time.Sleep(time.Second)
                }
        } else {
                cmd := exec.Command(os.Args[0], "child")
                cmd.SysProcAttr = &syscall.SysProcAttr{Setsid: true}
                log.Printf("child exited: %v", cmd.Run())
        }
}
$ go build
$ ./exectest
^Z
[1]+  Stopped                 ./exectest
$ bg
[1]+ ./exectest &
$ ps -ef | grep exectest | grep -v grep | grep -v vim
snowm     7914  5650  0 23:44 pts/7    00:00:00 ./exectest
snowm     7916  7914  0 23:44 ?        00:00:00 ./exectest child
$ kill -INT 7914 # kill parent process
[1]+  Exit 2                  ./exectest
$ ps -ef | grep exectest | grep -v grep | grep -v vim
snowm     7916     1  0 23:44 ?        00:00:00 ./exectest child

Note that the child process is still alive after parent process was killed. However, if I start the main process from systemd like this...

[snowm@localhost exectest]$ cat /etc/systemd/system/exectest.service 
[Unit]
Description=ExecTest

[Service]                        
Type=simple
ExecStart=/home/snowm/src/exectest/exectest
User=snowm

[Install]
WantedBy=multi-user.target
$ sudo systemctl enable exectest
ln -s '/etc/systemd/system/exectest.service' '/etc/systemd/system/multi-user.target.wants/exectest.service'
$ sudo systemctl start exectest

... then the child also dies when I kill the main process:

$ ps -ef | grep exectest | grep -v grep | grep -v vim
snowm     8132     1  0 23:55 ?        00:00:00 /home/snowm/src/exectest/exectest
snowm     8134  8132  0 23:55 ?        00:00:00 /home/snowm/src/exectest/exectest child
$ kill -INT 8132
$ ps -ef | grep exectest | grep -v grep | grep -v vim
$

How can I make the child survive?

Running go version go1.4.2 linux/amd64 under CentOS Linux release 7.1.1503 (Core).

monkey snow
  • 581
  • 1
  • 4
  • 10
  • SystemD probably kills everything in the process group (or cgroup, I haven't looked at the systemd setup in a while). Are you actually trying to emulate daemon behavior, or is this just for an example? – JimB Aug 25 '15 at 16:20
  • @JimB: My actual code spawns child processes that run ffmpeg. I'm basically using the pattern above to emulate daemon behavior, yes. – monkey snow Aug 25 '15 at 16:32
  • Why not leave the parent running, and let systemd manage it normally? I think there might be a way to spawn the child and notify system of the new process, but the former would be much easier. – JimB Aug 25 '15 at 16:41
  • @JimB: The child processes are expensive to restart, so they must not die when main process is restarted or crashes. However, thanks to your cgroup hint I found that the solution is to say `KillMode=process` (defaults to `control-group`). – monkey snow Aug 25 '15 at 16:54

3 Answers3

37

Solution is to add

KillMode=process

to the service block. Default value is control-group which means systemd cleans up any child processes.

From man systemd.kill

KillMode= Specifies how processes of this unit shall be killed. One of control-group, process, mixed, none.

If set to control-group, all remaining processes in the control group of this unit will be killed on unit stop (for services: after the stop command is executed, as configured with ExecStop=). If set to process, only the main process itself is killed. If set to mixed, the SIGTERM signal (see below) is sent to the main process while the subsequent SIGKILL signal (see below) is sent to all remaining processes of the unit's control group. If set to none, no process is killed. In this case, only the stop command will be executed on unit stop, but no process be killed otherwise. Processes remaining alive after stop are left in their control group and the control group continues to exist after stop unless it is empty.

monkey snow
  • 581
  • 1
  • 4
  • 10
5

The only way I know to solve this is to launch the child process with the --scope argument.

systemd-run --user --scope firefox

KillMode has been mentioned here also, but changing the KillMode also means that if your main process crashes, systemd won't restart it if any child process is still running.

Albin
  • 2,410
  • 2
  • 29
  • 27
  • Thank you! This worked for me and it's better than the accepted answer (at least for me) because I don't need to change systemd KillMode. – RandomGuy Jan 17 '23 at 17:52
0

If you cannot (like me) to change the KillMode of the service for some reason, you could try the at command (see man).

You can schedule your command to run 1 minute ahead. See an example:

# this will remove all .tmp files from "/path/" in 1 minute ahead (this task will run once)
echo rm /path/*.tmp | at now + 1 minute
Beto Neto
  • 3,962
  • 7
  • 47
  • 81