Unable to access docker-compose containers created inside docker

Question

I have a docker-compose.yml file that starts up a simple HTTP echo service on port 8800.

version: '2'

services:
    echo-server:
        image: luisbebop/echo-server
        container_name: echo-server
        ports:
        - "8800:8800"

Super simple stuff. If I run docker-compose up and run on my local machine:

echo "Hello World" | nc 127.0.0.1 8800

Then I see the echo. However: If I run this same compose scenario inside a docker container, through the GitLab runner, it fails.

I've tried to garner attention for this issue at GitLab here, but with limited results: https://gitlab.com/gitlab-org/gitlab-ce/issues/26566

My .gitlab-ci.yml file looks like:

---

stages:
  - docker_test

services:
  - docker:dind

docker_test:
  stage: docker_test
  image: docker:latest
  script:
  - docker version
  - apk update
  - apk add py-pip
  - pip install docker-compose
  - docker-compose up -d
  - sleep 10
  - netstat -tulpn
  - docker-compose port echo-server 8800
  - echo "Hello world" | nc 127.0.0.1 8800

And the output of gitlab-ci-multi-runner exec docker --docker-privileged docker_test is:

$ netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name

$ docker-compose port echo-server 8800
0.0.0.0:8800

$ echo "Hello world" | nc 127.0.0.1 8800
ERROR: Build failed: exit code 1

So it seems like these ports aren't available inside the docker container, which is in charge of the docker-compose services. Am I doing something wrong?

I do not want the ports exposed at the host level, rather I just want the ports opened by the docker container running the build, available to that container.

Host (My Mac) -> GitLab CI Container -> Docker Compose Container exposing 8800

  ^ Not here        ^ I want port 8800 accessible here

Edit: Further, if I attach a shell to the running CI container, I see:

/ # docker ps
CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS                    NAMES
ab192edf5872        luisbebop/echo-server   "/opt/echo-server/..."   2 minutes ago       Up About a minute   0.0.0.0:8800->8800/tcp   echo-server
/ # netstat -nltup
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name

So it "sees" the container, and the port mappings look okay, but the port isn't actually listening.

This thing is that the "inside" docker still runs on the outside docker engine - so the port bind actually binds to the localhost of the "root" host machine. However you should be able to specify docker network to use in the docker-compose file and set it to what docker network is attached to the GitLab container. The you will be able to connect to `nc echo-server 8800` — Lukáš Doležal, Jul 29 '17 at 21:59
to my previous comment, most likely the GitLab container will use `bridge` network. Add `network_mode: bridge` to the `echo-server` in your docker-compose.yml. Then change the commands to `nc echo-server 8800`. It may still not resolve the domain name. In that case you can get the IP via `$( docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' echo-server )` — Lukáš Doležal, Jul 29 '17 at 22:11
Hi @LukášDoležal, I'm not entirely sure that's the case. If I run `docker ps` at the "root" host machine, I see two containers - the `docker:dind` service, and the `docker:latest` image. If I run `docker ps` inside the `docker:dind` container, I see the three containers started by `docker-compose` - so the two sets of containers seem independent. Further, I cannot access the web service on `8800` from the "root" host (my Mac) even though I know it's running. (But I can if I just run `docker-compose` manually, as seen at the top of the question.) — Craig Otis, Jul 30 '17 at 01:37
did you check if nc command is present in the `docker:dind` or the `docker:latest` image? This error may be because of that. Normally nc doesn't come installed in images — Tarun Lalwani, Jul 30 '17 at 06:12
@TarunLalwani The `nc` command does come pre-installed in `docker:latest`, which is where I'm running it. — Craig Otis, Jul 30 '17 at 11:25

Robert · Accepted Answer · 2018-11-29T10:26:38.097

16

The container is running on the "docker host" which is, in your case, the other container that is supporting the Gitlab build:

services:
  - docker:dind

If I'm not wrong, its host name is docker. So access as this:

echo "Hello world" | nc docker 8800

To figure out what is the host of the docker daemon, use this:

script:
  - echo $DOCKER_HOST

edited Nov 29 '18 at 10:26

answered Jul 31 '17 at 13:05

Robert

33,429
8
90
94

5

That did it. You're 2 for 2 on my recent Docker questions. :-) Bounty in ~23 hours. – Craig Otis Jul 31 '17 at 13:26
$ echo $DOCKER_HOST tcp://docker:2375 – Astronaut Aug 01 '19 at 15:34
3

Cool, but what to I have to do if I require to http-connect to the started docker (`docker run --detach -p 8000:8000`)? `$DOCKER_HOST` is showing `unix:///var/run/docker.sock`. Connecting to `http://docker:8000` or `unix:///var/run/docker.sock:8000` fails. Everything runs under GitLab-CI. – Dmitriy Popov Feb 20 '20 at 17:17

Unable to access docker-compose containers created inside docker

1 Answers1

Linked