113

Right now our Jenkins agents generate a docker-compose.yml for each of our Rails projects and then run docker-compose up. The docker-compose.yml has a main "web" container that has rbenv and all of our other Rails dependencies inside. It is linked to a DB container that contains the test Postgres DB.

The problem comes when we need to actually run the tests and generate exit codes. Our CI server will only deploy if the test script returns exit 0, but docker-compose always returns 0, even if one of the container commands fail.

The other issue is that the DB container runs indefinitely, even after the web container is done running the tests, so docker-compose up never returns.

Is there a way we can use docker-compose for this process? We would need to be able to run the containers, but exit after the web container is complete and return it's exit code. Right now we are stuck manually using docker to spin up the DB container and run the web container with the --link option.

Logan Serman
  • 29,447
  • 27
  • 102
  • 141

9 Answers9

103

Since version 1.12.0, you can use the --exit-code-from option.

From documentation:

--exit-code-from SERVICE

Return the exit code of the selected service container. Implies --abort-on-container-exit.

panjan
  • 1,269
  • 1
  • 8
  • 9
  • 2
    That should be the right way of doing it if you are using ```docker-compose``` 1.12.0 and above. Maybe it is your case too. An example could be: ```docker-compose up --exit-code-from test-unit```. Note that it didn't work for me until I added a ```set -e``` at the beginning of my script. – Adrian Antunez Jul 27 '17 at 09:23
  • 3
    `--exit-code-from` doesn't work with `-d` though. It will throw these errors: `using --exit-code-from implies --abort-on-container-exit` and `--abort-on-container-exit and -d cannot be combined.` – ericat Aug 29 '17 at 16:06
  • 3
    I was able to get this working on Travis CI: https://travis-ci.org/coyote-team/coyote/builds/274582053 here's the travis.yml: https://github.com/coyote-team/coyote/blob/master/.travis.yml#L12 – subelsky Sep 12 '17 at 12:36
  • 3
    the documentation is atrocious. what flags is this compatible with? is it only one service or can you pass it several? – worc Nov 14 '18 at 19:23
  • 1
    Note that these args apply to [docker-compose up](https://docs.docker.com/compose/reference/up/) and not [docker-compose run](https://docs.docker.com/compose/reference/run/) – Ed Randall Nov 23 '21 at 14:48
  • note that to run only certain service u should use command `set -e; docker-compose -f file up --exit-code-from service service` – Demetry Pascal May 25 '23 at 11:07
49

docker-compose run is the simple way to get the exit statuses you desire. For example:

$ cat docker-compose.yml 
roit:
    image: busybox
    command: 'true'
naw:
    image: busybox
    command: 'false'
$ docker-compose run --rm roit; echo $?
Removing test_roit_run_1...
0
$ docker-compose run --rm naw; echo $?
Removing test_naw_run_1...
1

Alternatively, you do have the option to inspect the dead containers. You can use the -f flag to get just the exit status.

$ docker-compose up
Creating test_naw_1...
Creating test_roit_1...
Attaching to test_roit_1
test_roit_1 exited with code 0
Gracefully stopping... (press Ctrl+C again to force)
$ docker-compose ps -q | xargs docker inspect -f '{{ .Name }} exited with status {{ .State.ExitCode }}'
/test_naw_1 exited with status 1
/test_roit_1 exited with status 0

As for the db container that never returns, if you use docker-compose up then you will need to sigkill that container; that's probably not what you want. Instead, you can use docker-compose up -d to run your containers daemonized, and manually kill the containers when your test is complete. docker-compose run should run linked containers for you, but I have heard chatter on SO about a bug preventing that from working as intended right now.

David Resnick
  • 4,891
  • 5
  • 38
  • 42
kojiro
  • 74,557
  • 19
  • 143
  • 201
  • The problem with docker run is that it does not give any output when run with -T, and we want the output so we can inspect failed builds. – Logan Serman Apr 10 '15 at 19:45
  • 1
    @LoganSerman you can inspect the output with `docker-compose logs` – kojiro Apr 10 '15 at 19:47
  • Is there a way to constantly pipe those logs to STDOUT during the run so we can see it while the CI build is in progress? – Logan Serman Apr 10 '15 at 19:54
  • I guess I don't understand why you are running with `-T` – kojiro Apr 10 '15 at 20:21
  • Some of the commands we run inside of the container to run tests have the potential to ask for input, we want to run with -T to avoid this. Rbenv for example asks if you want to reinstall a Ruby version if it already exists. – Logan Serman Apr 15 '15 at 14:42
27

Building on kojiro's answer:

docker-compose ps -q | xargs docker inspect -f '{{ .State.ExitCode }}' | grep -v '^0' | wc -l | tr -d ' '

  1. get container IDs
  2. get last runs exit code for each container ID
  3. only status codes that does not start with '0'
  4. count number of non-0 status codes
  5. trim out white space

Returns how many non-0 exit codes were returned. Would be 0 if everything exited with code 0.

Lukas Würzburger
  • 6,543
  • 7
  • 41
  • 75
spenthil
  • 3,373
  • 29
  • 17
  • You can also use the non-quiet output from `docker-compose ps`, for example: `docker-compose ps | grep -c "Exit 1"` will give you the count where "Exit 1" is matched in the display from `docker-compose ps` (which provides a pretty-printed summary table of results). The exit codes are listed in the "State" column. – eharik Nov 03 '15 at 19:47
  • This is really awesome. In my case failing test suite running in containers don't make the containers exit with a code of 1. I can't aggregate if any exited with a code of 1 since none of them do.... Any idea how to handle this case? – walkerrandophsmith Sep 30 '18 at 05:30
15

Use docker wait to get the exit code:

$ docker-compose -p foo up -d
$ ret=$(docker wait foo_bar_1)

foo is the "project name". In the example above, I specified it explicitly, but if you don't supply it, it's the directory name. bar is the name you give to the system under test in your docker-compose.yml.

Note that docker logs -f does the right thing, too, exiting when the container stops. So you can put

$ docker logs -f foo_bar_1

between the docker-compose up and the docker wait so you can watch your tests run.

Bryan Larsen
  • 9,468
  • 8
  • 56
  • 46
15

--exit-code-from SERVICE and --abort-on-container-exit don't work in scenarios where you need to run all containers to completion, but fail if one of them exited early. An example might be if running 2 test suits in concurrently in different containers.

With @spenthil's suggestion, you can wrap docker-compose in a script that will fail if any containers do.

#!/bin/bash
set -e

# Wrap docker-compose and return a non-zero exit code if any containers failed.

docker-compose "$@"

exit $(docker-compose -f docker-compose.ci.build.yml ps -q | tr -d '[:space:]' |
  xargs docker inspect -f '{{ .State.ExitCode }}' | grep -v 0 | wc -l | tr -d '[:space:]')

Then on your CI server simply change docker-compose up to ./docker-compose.sh up.

Matt Cole
  • 2,491
  • 17
  • 21
  • 1
    this script never reaches the exit section as other containers (such as databases, web apps) run perpetually. running in detached mode it exits as soon as the container are up – Baldy Mar 27 '18 at 16:22
  • That's right, this only works if you want to run _all_ containers to completion. Probably not particularly common, but it was useful for me at the time of writing and I thought I would share it. – Matt Cole Mar 27 '18 at 16:29
  • Upvoted your answer anyway as it got me most of the way there! Adding Docker wait on each test container in detached mode got it working. Thanks for sharing:) – Baldy Mar 27 '18 at 19:59
11

If you're willing to use docker-compose run to manually kick off your tests, adding the --rm flag, oddly enough, causes Compose to accurately reflect your command's exit status.

Here's my example:

$ docker-compose -v
docker-compose version 1.7.0, build 0d7bf73

$ (docker-compose run bash false) || echo 'Test failed!'  # False negative.

$ (docker-compose run --rm bash false) || echo 'Test failed!'  # True positive.
Test failed!

$ (docker-compose run --rm bash true) || echo 'Test failed!'  # True negative.
esmail
  • 517
  • 11
  • 9
2

docker-rails allows you to specify which container's error code is returned to the main process, so you CI server can determine the result. It is a great solution for CI and development for rails with docker.

For example

exit_code: web

in your docker-rails.yml will yield the web containers exit code as a result of the command docker-rails ci test. docker-rails.yml is just a meta wrapper around the standard docker-compose.yml that gives you the potential to inherit/reuse the same base config for different environments i.e. development vs test vs parallel_tests.

kross
  • 3,627
  • 2
  • 32
  • 60
2

You can see exit status with:

echo $(docker-compose ps | grep "servicename" | awk '{print $4}')
Max Alibaev
  • 681
  • 7
  • 17
RT Bathula
  • 169
  • 1
  • 2
  • 9
  • Thanks for getting this started. Here's my version of this one (which works better for me b/c I think the command output format has changed since this answer was written) – `docker-compose ps | grep servicename | grep -v 'Exit 0' && echo "Automation or integration tests failed." && exit 1` – DTrejo Apr 03 '19 at 23:39
2

In case you might run more docker-compose services with same name on one docker engine, and you don't know the exact name: 

docker-compose up -d
(exit "${$(docker-compose logs -f test-chrome)##* }")

echo %? - returns exit code from test-chrome service

Benefits:

  • wait's for exact service to exit
  • uses service name, not container name
cvakiitho
  • 1,377
  • 13
  • 27