0

I'm building a docker image for running Gitlab CI jobs. One of the components needs systemd up and running inside the container, this is not trivial but there are several guides on the web so I managed to do it. Part of the process requires to define this entrypoint in the Dockerfile:

ENTRYPOINT ["/usr/sbin/init"]

so that systemd runs as PID 1 in the container, as needed. This seems to conflict with Gitlab CI requirements: as far as I understand gitlab-runner overrides the Dockerfile's CMD to spawn a shell which then executes the CI script. But the /usr/sbin/init entrypoint cannot understand the Gitlab's CMD so the shell is not spawned and the execution halts.

I cannot figure out how to solve this:

  • executing an entrypoint script which starts /usr/sbin/init and then a shell won't work because systemd won't be PID1;
  • using a shell as ENTRYPOINT and then systemd as CMD won't work since Gitlab CI overrides CMD.

I cannot think of any other possible solution, so any help is much appreciated.

Nicola Mori
  • 777
  • 1
  • 5
  • 19
  • It's OK if systemd is not PID1. – sytech Sep 15 '22 at 04:18
  • If I don't start /usr/sbin/init in ENTRYPOINT but e.g. in a script used as ENTRYPOINT then I get: "Failed to get D-Bus connection: Operation not permitted" every time I launch systemctl, for example `systemctl list-units` returns that error. – Nicola Mori Sep 15 '22 at 09:18
  • @sytech I investigated a bit and I think you refer to systemd's user mode: as far as I understand it works only if the system has been booted with systemd, i.e. is there is a global systemd instance with PID 1. Am I wrong? – Nicola Mori Sep 15 '22 at 14:16

3 Answers3

1

You can always fork a bash shell to do the work and then exec systemd with pid 1 like this:

ENTRYPOINT [ "/bin/bash", "-c", "nohup bash -c \"${@} ; /sbin/init 0\" & exec /sbin/init", "${@}" ]
Ant
  • 11
  • 1
  • That could work, but another requirement is that PID 1 must terminate as the CI job is finished, otherwise the CI container never terminates and the job times out. I figured out a complete solution that I'm going to post soon. – Nicola Mori Dec 30 '22 at 08:47
0

Finally I have been able to put together a working solution. The ENTRYPOINT executes a script:

ENTRYPOINT ["/entrypoint.sh"]

which retrieves the stdin/stdout fds of PID 1 (i.e. those that will be used by Gitlab CI for the job I/O), pins them by attaching them to a long-running process and then spawns systemd as PID 1:

#!/bin/bash

# Start a long-running process to keep the container pipes open
sleep infinity < /proc/1/fd/0 > /proc/1/fd/1 2>&1 &
# Wait a bit before retrieving the PID
sleep 1
# Save the long-running PID on file
echo $! > /container-pipes-pid
# Start systemd as PID 1
exec /usr/lib/systemd/systemd

The pinning of stdin/stdout is needed since some systemd versions close stdin/stdout at startup, but they are needed since the CI infrastructure uses these to send CI commands to the shell and receive console output. So attaching them to sleep infinity makes them persist even after exec /usr/lib/systemd/systemd.

The shell is then spawned by a systemd unit (previously enabled in the Dockerfile):

[Unit]
Description=Start bash shell attached to container STDIN/STDOUT

[Service]
Type=simple
PassEnvironment=PATH LD_LIBRARY_PATH
ExecStart=/bin/bash -c "echo Attaching to pipes of PID `cat container-pipes-pid` && exec /bin/bash < /proc/`cat container-pipes-pid`/fd/0 > /proc/`cat container-pipes-pid`/fd/1 2>/proc/`cat container-pipes-pid`/fd/2"
ExecStopPost=/usr/bin/systemctl exit $EXIT_STATUS

[Install]
WantedBy=multi-user.target rescue.target

The pinned fds are attached to the shell (no conflict is generated by this since neither systemd nor sleep do I/O on those fds), so the shell correctly receives CI commands and directs console output to CI logs. Then at the end of the job, when stdin is closed by the CI infrastructure, the shell terminates and the unit shuts down the container, returning the job execution result so that the CI correctly retrieves the job outcome.

This is a rather involved implementation which can for sure be refined and improved, but is working beautifully for my purpose.

Nicola Mori
  • 777
  • 1
  • 5
  • 19
0

I have replicated the solution posted by Nicola Mori and it is works fine, unamended, for Debian Bookworm and Trixie and Ubuntu Jammy -- good work!

For Debian Bullseye the path for systemd is /lib/systemd/systemd rather than /usr/lib/systemd/systemd so that needs changing.

I saved the systemd unit as posted above as bash.service and implemented adding it and the entrypoint.sh in the Docker file as follows:

COPY bash.service /etc/systemd/system/bash.service
RUN chown root:root /entrypoint.sh \
    && chmod 755 /entrypoint.sh \
    && chown root:root /etc/systemd/system/bash.service \
    && chmod 644 /etc/systemd/system/bash.service \
    && systemctl enable bash.service
ENTRYPOINT ["/entrypoint.sh"]

Note that if these containers are run on in non-privileged mode then they fail with:

ERROR: Job failed: exit code 255

I don't know if a check if the container is running in privileged mode could be added to the entrypoint.sh script or if a check might interfere with the PID assignment?

I have also posted about this on the GitLab Discourse forum.

Chris Croome
  • 11
  • 3