1

I have a docker file with two services running on the same network. The purpose is to scrape data and store it in a database.

When I attempt to run the program db_connector.py inside service block_scraper I get an error (Timeout):

pymongo.errors.ServerSelectionTimeoutError: storage:27017: [Errno 111] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 64f24e6e941b3aa3dea67cc9, topology_type: Unknown, servers: [<ServerDescription ('storage', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('storage:27017: [Errno 111] Connection refused')>]>

As seen in the error, my most recent attempts are using the Docker service name, storage, to refer to the service.

I was able to get this to work fine when running both services natively outside docker. How can I get them to talk to eachother inside docker?

Project Tree:
|-Dockerfile
|-docker-compose.yml
|-requirements.txt
|-app/
|-|-main.py
|-|-db_connector.py

The docker-compose.yml file:

version: "3.2"

services:

  storage:
    image: mongo:7.0.1-rc0
    container_name: mongodb
    restart: unless-stopped
    ports:
      - 27017-27019:27017-27019
    volumes:
      - ./data:/data/db
    networks:
      - scraper_network

  block_scraper:
    build: .
    depends_on:
      - storage
    volumes:
      - ./app:/app
    environment:
      - STORAGE_CONNECTION_STR=mongodb://storage:27017
    networks:
      - scraper_network

networks:
  scraper_network:

The Dockerfile (for block_scraper service):

# Use the official Python 3.11 image as a parent image
FROM python:3.11-slim-buster

# Set the working directory inside the container
WORKDIR /app

# Copy in and install requirements
COPY ./requirements.txt /app/requirements.txt
RUN pip install -Ur requirements.txt

# Copy the application code into the container
COPY . /app

# Specify the command to run your main program
CMD ["python", "main.py"]

The db_connector.py file:

import os
import pymongo

# Create a single MongoClient instance and reuse it across calls to get_collection
mongo_host = os.environ.get("STORAGE_CONNECTION_STR")
client = pymongo.MongoClient(mongo_host)

def get_collection(mongo_db_name="blocks101", mongo_collection_name="block"):
    database = client[mongo_db_name]
    return database[mongo_collection_name]

def close():
    client.close()

For simplicity, lets say this is the main.py program:

from db_connector import get_collection

get_collection()
MrChadMWood
  • 113
  • 11

1 Answers1

0

Using mongodb:/<neighbour_container_name>:27017 is the only correct way to connect. Other listed variants will not work, because from inside docker container localhost refers to the container itself (for connection to container's host address there's host.docker.internal)

It seems you should set container_name: storage (or remove this line completely) also.

upd:

I have successfully reproduced your setup and connected to default database as a result:

root@85081a0c46fc:/app# python3
>>> import os
>>> import pymongo
>>> mongo_host = os.environ.get("STORAGE_CONNECTION_STR")
>>> client = pymongo.MongoClient(mongo_host)
>>> client
MongoClient(host=['storage:27017'], document_class=dict, tz_aware=False, connect=True)

Are you able to connect to mongo container from host?

zrthmnk
  • 36
  • 3
  • Unfortunately, this didn't resolve the issue for me. I've updated my Q to include more information. – MrChadMWood Sep 01 '23 at 21:10
  • 1
    I am able to connect too now... but never the first time. I think Mongo isn't ready to accept connections before `block_scraper` attempts -- which is strange in consideration to my `depends_on` clause inside `docker-compose.yml`. Is there any way to add a `healthcheck` in `docker-compose` specific for when Mongo is ready to accept external connections? – MrChadMWood Sep 01 '23 at 21:56
  • yes, you can implement healthcheck for container and use --wait flag - similar problem (and solution) was discussed here: https://stackoverflow.com/questions/67805969/wait-for-specific-container-when-using-docker-compose-up-d – zrthmnk Sep 02 '23 at 09:33