How does one connect two services in the local docker-compose network?

Question

I have followed the instructions, I think, and have come up with the following configuration:

 version: '3.9'
 services:
   flask:
    image: ops:imgA
      ports:
       - 5000:5000
      volumes:
       - /opt/models:/opt/models
      entrypoint: demo flask
    streamlit:
      image: ops:imgB
      ports:
        - 8501:8501
      entrypoint: streamlit run --server.port 8501 demo -- stream --flask-hostname flask

The --flask-hostname flask sets the host name used in an http connect, i.e.: http://flask:5000. I can set it to anything.

The basic problem here is that I can spin up one of these images, install tmux, and run everything within a single image.

But, when I split it across multiple images and use docker-compose up (which seems better than tmux), the containers can't seem to connect to each other.

I have rattled around the documentation on docker's website, but I've moved on to the troubleshooting stage. This seems to be something that should "just work" (since there are few questions along these lines). I have total control of the box I am using, and can open or close whatever ports needed.

Mainly, I am trying to figure out how to allow, with 100% default settings nothing complicated, these two services (flask and streamlit) to speak to each other.

There must be 1 or 2 settings that I need to change, and that is it.

Any ideas?

Update

I can access all of the services externally, so I am going to open up external connections between the services (using the external IP) as a "just work" quick fix, but obviously getting the composition to work internally would be the best option.

I have also confirmed that the docker-compose and docker versions are up to date.

Update-2: changed from flask@127.0.0.1 to flask@0.0.0.0

Flask output:

flask_1      |  * Serving Flask app "flask" (lazy loading)
flask_1      |  * Environment: production
flask_1      |    WARNING: This is a development server. Do not use it in a production deployment.
flask_1      |    Use a production WSGI server instead.
flask_1      |  * Debug mode: on
flask_1      | INFO:werkzeug: * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
flask_1      | 2020-12-19 02:22:16.449 INFO    werkzeug:  * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
flask_1      | INFO:werkzeug: * Restarting with inotify reloader
flask_1      | 2020-12-19 02:22:16.465 INFO    werkzeug:  * Restarting with inotify reloader
flask_1      | WARNING:werkzeug: * Debugger is active!
flask_1      | 2020-12-19 02:22:22.003 WARNING werkzeug:  * Debugger is active!

Streamlit:

streamlit_1  |
streamlit_1  |   You can now view your Streamlit app in your browser.
streamlit_1  |
streamlit_1  |   Network URL: http://172.18.0.3:8501
streamlit_1  |   External URL: http://71.199.156.142:8501
streamlit_1  |
streamlit_1  | 2020-12-19 02:22:11.389 Generating new fontManager, this may take some time...

And the streamlit error message:

ConnectionError: 

 HTTPConnectionPool(host='flask', port=5000): 
    Max retries exceeded with url: /foo/bar 

(Caused by NewConnectionError(
  '<urllib3.connection.HTTPConnection object at 0x7fb860501d90>: 
     Failed to establish a new connection: 
       [Errno 111] Connection refused'
  )
)

Update-3: Hitting refresh fixed it.

What you have looks fine. (As described in [Networking in Compose](https://docs.docker.com/compose/networking/), you can use the service name `flask` as a host name.) What error are you getting? — David Maze, Dec 19 '20 at 00:52
@DavidMaze It looks like I am getting an error 111: Connection Refused — Chris, Dec 19 '20 at 01:07
In `ops:imgA`, what address is the service listening on? (It needs to be listening on `0.0.0.0`; if it's listening on `127.0.0.1` it won't be reachable from outside its container, but it would work with your previous setup where both processes are in the same container.) — David Maze, Dec 19 '20 at 01:37
(That you're getting "connection refused" and not, say, "host not found" suggests you have the core Docker networking set up correctly, and the `streamlit` container is correctly resolving the `flask` host name.) — David Maze, Dec 19 '20 at 01:40
@DavidMaze it is using 127.0.0.1 -- that is correct. I need to set it to be 0.0.0.0? — Chris, Dec 19 '20 at 02:16
@DavidMaze I have changed flask to listen on 0.0.0.0; the streamlit app is looking for "flask" — Chris, Dec 19 '20 at 02:24
You do. (The usual symptom of this is that you get a "connection refused" type error making a call from outside Docker, and changing to 0.0.0.0 will fix that as well.) The `streamlit` container still needs to connect to the host name `flask`, that doesn't change. — David Maze, Dec 19 '20 at 02:24

score 1 · Accepted Answer · answered Dec 19 '20 at 11:30

The server process must be listening on the special "all interfaces" address 0.0.0.0. Many development-type servers by default listen on "localhost only" 127.0.0.1, but in Docker each container has its own private notion of localhost. If you use tmux or docker exec to run multiple processes inside a container, they have the same localhost and can connect to each other, but if the client and server are running in different containers, the request doesn't arrive on the server's localhost interface, and if the server is listening on "localhost only" it won't receive it.

Your setup is otherwise correct, with only the docker-compose.yml you include in the question. Some other common problems:

You must connect to the port the server process is listening on inside the container. If you remap it externally with ports:, that's ignored, and you'd connect to the second ports: number. Correspondingly, ports: aren't required. (expose: also isn't required and doesn't do anything at all.)
The client may need to wait for the server to start up. If the client depends_on: [flask] the host name will usually resolve (unless the server dies immediately) but if it takes a while to start up you will still get "connection refused" errors. See Docker Compose wait for container X before starting Y.
Neither container may use network_mode: host. This disables Docker's networking features entirely.
If you manually declare networks:, both containers need to be on the same network. You do not need to explicitly create a network for inter-container communication to work: Compose provides a default network for you, which is used if nothing else is declared.
Use the Compose service names as host names. You don't need to explicitly specify container_name: or links:.

How does one connect two services in the local docker-compose network?

1 Answers1