Why can't I stop my Mojo app using systemd?

Question

I can fire up my Mojo app via systemd without any issues, but I can't stop or restart it in the same way. I've poached my config mostly from the Mojo docs at http://mojolicious.org/perldoc/Mojolicious/Guides/Cookbook#Pre-forking:

[Unit]
Description=ldsquiz.org
After=network.target

[Service]
Type=simple
Restart=always
RestartSec=30
ExecStart=/home/sites/www.ldsquiz.org/checkout/bin/carton-exec app.pl prefork -m production -l http://*:5103
KillMode=process
User=ldsquiz
Group=ldsquiz
SyslogIdentifier=ldsquiz-org

[Install]
WantedBy=multi-user.target

If I stop it via sudo service ldsquiz-org stop then syslog says Jan 13 16:37:48 web-hosting systemd[1]: Stopped ldsquiz.org. but the processes are still running.

A restart via systemd looks like I would expect, since the listen socket is still in use because of the failed shutdown:

Jan 13 16:39:13 web-hosting systemd[1]: Stopped ldsquiz.org.
Jan 13 16:39:13 web-hosting systemd[1]: Started ldsquiz.org.
Jan 13 16:39:14 web-hosting ldsquiz-org[20353]: Can't create listen socket: Address already in use at 
/home/sites/www.ldsquiz.org/checkout/local/lib/perl5/Mojo/IOLoop.pm line 126.
Jan 13 16:39:14 web-hosting systemd[1]: ldsquiz-org.service: Main process exited, code=exited, status=98/n/a
Jan 13 16:39:14 web-hosting systemd[1]: ldsquiz-org.service: Unit entered failed state.
Jan 13 16:39:14 web-hosting systemd[1]: ldsquiz-org.service: Failed with result 'exit-code'.
Jan 13 16:39:15 web-hosting ldsquiz-org[16886]: [Sat Jan 13 16:39:15 2018] [info] Creating process id file "/tmp/prefork.pid"

What am I missing here?

I usually control my `systemd` units with `systemctl start/stop/status/restart $serviceName`. Does that make any difference? I.e. `systemctl stop ldsquiz-org`? — PerlDuck, Jan 13 '18 at 17:19
@PerlDuck that seems to have the same problem for me. It thinks that the app has been stopped but the original processes are still hanging around. — oalders, Jan 13 '18 at 18:55
I know the docs you refer to say `Type=simple` but this `prefork` parameter makes me doubt. Perhaps give `Type=forking` a try. See [here](https://www.freedesktop.org/software/systemd/man/systemd.service.html#Type=) for the possible values. Perhaps you also need to play a bit with the `KillMode=` setting. See [here](https://www.freedesktop.org/software/systemd/man/systemd.kill.html#KillMode=) for the possible values. And remember to call `systemctl daemon-reload` whenever you fiddle with the unit file. — PerlDuck, Jan 13 '18 at 19:01
@PerlDuckwonderful! `KillMode=control-group` did the trick for me. Looks like I'll need to see about a Mojo doc patch. If you'd like to write an answer I'm happy to accept it. — oalders, Jan 13 '18 at 19:11
As an addendum, this issue has now been addressed in the Mojo docs: https://github.com/kraih/mojo/commit/393daba639d1a4d3f3c07c4094425a35a78889ca — oalders, Feb 05 '18 at 20:21

score 8 · Accepted Answer · answered Jan 13 '18 at 19:42

Apparently there's a flaw in the Mojolicious Cookbook w.r.t. preforking and systemd units.

They say:

And to manage the pre-forking web server with systemd, you can use a unit configuration file like this.
[Unit]
Description=My Mojolicious application
After=network.target

[Service]
Type=simple
ExecStart=/home/sri/myapp/script/my_app prefork -m production -l http://*:8080
KillMode=process

[Install]
WantedBy=multi-user.target

The KillMode=process setting turns out to be wrong and needs to be replaced with KillMode=control-group — which is the default setting, so we can drop it altogether.

The KillMode docs say:

If set to control-group, all remaining processes in the control group of this unit will be killed on unit stop (for services: after the stop command is executed, as configured with ExecStop=). If set to process, only the main process itself is killed.

^{(emphasis mine)}

Thus:

[Unit]
Description=My Mojolicious application
After=network.target

[Service]
Type=simple
ExecStart=/home/sri/myapp/script/my_app prefork -m production -l http://*:8080
KillMode=control-group

[Install]
WantedBy=multi-user.target

Note that the KillMode=control-group can be omitted as this setting is the default.

References:

Yes. That "KillMode" setting has since been removed from the example docs: https://github.com/mojolicious/mojo/pull/1183 — Kevin G., Jul 16 '20 at 16:09

Why can't I stop my Mojo app using systemd?

1 Answers1