2

The tasks seems simple. When the vm is done rebooting, the podman containers should be started again. Both containers spin up, but then exit because of SIGTERM.

We used this command on the vm running Ubuntu 22.04.2 LTS

podman-compose -f projects/04_floof/podman-compose.yml down && podman-compose -f projects/04_floof/podman-compose.yml up -d

Because of the error, we have tried several approaches. Those with crontab all with the same outcome, as shown below

  • inside crontab
  • inside crontab with nohup
  • inside crontab with disown
  • inside crontab with screen
  • using systemd

Here's the podman-compose.yml:

version: "3.9"
services:
    04_floof:
        container_name: 04_floof
        image: docker.io/myrepo/04_floof:latest
        networks:
            - floof-internal
    cf_04_floof:
        container_name: cf_04_floof
        image: docker.io/cloudflare/cloudflared:latest
        command: tunnel --no-autoupdate run --token XXXXXXXXXXXXX
        networks:
            - floof-internal
networks:
    floof-internal: {}

Here's the log of one container that shows it exits because of SIGTERM.

user@vm3:~$ podman logs 04_floof 
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: /etc/nginx/conf.d/default.conf differs from the packaged version
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2023/04/16 13:07:48 [notice] 1#1: using the "epoll" event method
2023/04/16 13:07:48 [notice] 1#1: nginx/1.23.3
2023/04/16 13:07:48 [notice] 1#1: built by gcc 10.2.1 20210110 (Debian 10.2.1-6) 
2023/04/16 13:07:48 [notice] 1#1: OS: Linux 5.15.0-69-generic
2023/04/16 13:07:48 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2023/04/16 13:07:48 [notice] 1#1: start worker processes
2023/04/16 13:07:48 [notice] 1#1: start worker process 23
2023/04/16 13:07:48 [notice] 1#1: start worker process 24
2023/04/16 13:11:06 [notice] 1#1: signal 15 (SIGTERM) received, exiting
2023/04/16 13:11:06 [notice] 23#23: signal 15 (SIGTERM) received, exiting
2023/04/16 13:11:06 [notice] 23#23: exiting
2023/04/16 13:11:06 [notice] 24#24: exiting
2023/04/16 13:11:06 [notice] 23#23: exit
2023/04/16 13:11:06 [notice] 24#24: exit
2023/04/16 13:11:06 [notice] 1#1: signal 15 (SIGTERM) received, exiting
2023/04/16 13:11:06 [notice] 1#1: signal 17 (SIGCHLD) received from 24
2023/04/16 13:11:06 [notice] 1#1: worker process 24 exited with code 0
2023/04/16 13:11:06 [notice] 1#1: signal 29 (SIGIO) received
2023/04/16 13:11:06 [notice] 1#1: signal 17 (SIGCHLD) received from 23
2023/04/16 13:11:06 [notice] 1#1: worker process 23 exited with code 0
2023/04/16 13:11:06 [notice] 1#1: exit
ratlan
  • 341
  • 2
  • 12

1 Answers1

1

As similar issue was raised with containers/podman issue 14531 in 2022.

Shutdown procedure hangs for ~2 minutes because podman can't be stopped. Then podman is killed and shutdown is complete.

If adding a stop_signal: SIGINT to each of your service does not help, the aforementioned issue references PR 16785

health check: ignore dependencies of transient systemd units/timers

When stopping the transient systemd timer/unit which powers running health checks, make sure to ignore its dependencies.
It turns out that we're otherwise running into a timeout when running a container in a systemd unit and reboot.

This is for podman 4.4.0+.

That would mean you do not need the podman down command, and can focus, in your systemd start service, on the podman-compose -f projects/04_floof/podman-compose.yml up -d part.

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • Not sure how this should help. Your answer seems to cover an issue regarding shutdown, but I'm having an issue when trying to starti the containers. – ratlan Apr 22 '23 at 08:21
  • @ratlan That is the idea: The `systemd` "start" service should focus only on starting the container instead of "stopping + starting" them. – VonC Apr 22 '23 at 17:16