1

I'm using different Docker images to run code and noticed the following:

$ time sudo docker stop $(sudo docker run -dit --rm centos:latest)
509285200bcf4ea8389219319b6c4af6554abf9f1c9d9ffb152cd109b6a21453

real    0m1,374s
user    0m0,087s
sys 0m0,027s
$ time sudo docker stop $(sudo docker run -dit --rm debian:latest)
221221b4b9238de633ca68d9539b024c06d015ea503a8b7fd5b827dc903193b8

real    0m1,345s
user    0m0,086s
sys 0m0,034s
$ time sudo docker stop $(sudo docker run -dit --rm ubuntu:latest)
5512fb2505f28f7d2a52788a2ed9775c78ae40805da44e51bb3f22073133f19c

real    0m1,341s
user    0m0,075s
sys 0m0,048s
$ time sudo docker stop $(sudo docker run -dit --rm alpine:latest)
dc1775fa2734c753a5ba6f3fc0b14bf356376d8c701fcf6efa637f1795f41b4a

real    0m11,439s
user    0m0,089s
sys 0m0,032s

How can this difference be explained?

Update: The time difference is solely produced by docker stop. If I extend the timeout with docker stop -t 30 for example, the alpine container takes all of that time and eventually times out, while the behavior of the other containers is naturally not affected by the increased timeout. This seems to have something to do with the propagation of SIGTERM - I can find some previous issues on this, but they are about docker stop in general and not regarding alpine explicitly. This doesn't explain why the issue exists for alpine and not for the other images.

finefoot
  • 9,914
  • 7
  • 59
  • 102
  • Very interesting finding! I can reproduce on Ubuntu 16.04.6. – valiano Mar 03 '20 at 12:15
  • Reproduced with: `Docker version 18.09.7, build 2d0083d`, `Docker version 19.03.6, build 369ce74a3c` – valiano Mar 03 '20 at 12:39
  • Sure thing @fineroot, and kodus on your findings. Also reproduced with different alpine images: `3.11`, `3.10` and `3.9`, and also with `gliderlabs/alpine:3.9` and `gliderlabs/alpine:3.7` (for which Dockerfiles are available) – valiano Mar 03 '20 at 13:44
  • I suspect the issue is related to musl-libc, used by Alpine, and its signal handling, but it's just a speculation. I've added the [signals] tag, hoping it would get to the right eyes. – valiano Mar 03 '20 at 15:20
  • Nice! Great find – valiano Mar 05 '20 at 06:50

2 Answers2

2

From the docs for docker stop:

The main process inside the container will receive SIGTERM, and after a grace period, SIGKILL.

As opposed to CentOS, Debian and Ubuntu which use bash, Alpine uses busybox sh which ignores SIGTERM and the container is only stopped after the timeout with SIGKILL.

That's different to current versions of bash which honor a SIGTERM and terminate. However, previous versions of bash did ignore SIGTERM, too: Using an older CentOS image like centos:7 produces a timeout as well.

In the end, it's not clear why bash currently honors SIGTERM (see Why does SIGTERM work to terminate bash but not ash or dash?) or why SIGHUP works for busybox sh on the native system but not with docker kill (see Why does SIGHUP not work on busybox sh in an Alpine Docker container?).

finefoot
  • 9,914
  • 7
  • 59
  • 102
1

If this is using alpine linux image with Python applications, it's known to take a longer time to build especially with bigger images.

One suggestion is to strip off the alpine base image installation to reduce slow speed times, so remove the originally installed packages or maybe add an option not to cache package downloads using a multi-stage build may work as well.

de_classified
  • 1,927
  • 1
  • 15
  • 19
  • The images in question are pure Linux base images. – valiano Mar 04 '20 at 08:13
  • 1
    @FishingCode Hey thanks! Yes, I read about that a lot, too, when researching the issue. I'm not sure it's related to the shell. But of course, it might be. Didn't look any further into that. :) – finefoot Mar 05 '20 at 10:42