0

I've a pipeline in GitLab CI:

image: docker:latest

services:
  - docker:dind


stages:
  - build

before_script:
  - apk add --update python3 python3-pip python3-dev git openssh-client && pip3 install docker-compose


main_script:
  stage: build
  network_mode: bridge
  script:
    - docker-compose up -d --build
    - docker-compose exec -T users python manage.py test
    - docker-compose exec -T users flake8 project --ignore=W191,E401,E999

after_script:
  - docker-compose down

docker-compose.yml:

version: '3.7'

services:

  users:
    build:
      context: ./services/users
      dockerfile: Dockerfile
    volumes:
      - './services/users:/usr/src/app'
    expose:
      - 5000
    environment:
      - FLASK_DEBUG=True
      - APP_SETTINGS=project.config.DevelopmentConfig
      - DATABASE_URL=postgresql://postgres:postgres@users-db:5432/users_dev
      - DATABASE_TEST_URL=postgresql://postgres:postgres@users-db:5432/users_test
    depends_on:
      - users-db

  users-db:
    build:
      context: ./services/users/project/db
      dockerfile: Dockerfile
    expose:
      - 5432
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres

  nginx:
    build:
      context: ./services/nginx
      dockerfile: Dockerfile

    restart: always
    ports:
      - 80:80
    depends_on:
      - users

PostgreSQL Dockerfile:

FROM postgres:14.7-alpine

ADD create.sql /docker-entrypoint-initdb.d

I bashed into the PostgreSQL container and I did see listen_addresses='*' in the configuration file, so that's fine, but how can I troubleshoot this further?

Error output:

sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) connection to server at "users-db" (172.19.0.2), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?

File entrypoint.sh

#!/bin/sh

echo "Waiting for postgres ..."

while ! nc -z users-db 5432; do
    sleep 0.1
done

echo "PostgreSQL started"

python3 manage.py run -h 0.0.0.0

netstat shows:

Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 runner--project-0:39744 docker:2375             TIME_WAIT
tcp        0      0 runner--project-0:38424 docker:2375             TIME_WAIT
tcp        0      0 runner--project-0:38456 docker:2375             TIME_WAIT
tcp        0      0 runner--project-0:40648 docker:2375             TIME_WAIT
tcp        0      0 runner--project-0:60706 ec2-52-1-184-176.co:443 TIME_WAIT
tcp        0      0 runner--project-0:38442 docker:2375             TIME_WAIT
tcp        0      0 runner--project-0:38474 docker:2375             TIME_WAIT
tcp        0      0 runner--project-0:57332 docker:2375             TIME_WAIT
tcp        0      0 runner--project-0:38460 docker:2375             TIME_WAIT
tcp        0      0 runner--project-0:57348 docker:2375             TIME_WAIT
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags       Type       State         I-Node   Path

I've added docker-compose exec -T users-db /bin/sh -c "echo 'host all all 0.0.0.0/0 md5' >> /var/lib/postgresql/data/postgresql.conf" to ensure to the main script in pipeline. It hasn't helped much.

ps ux | grep postgres gives:

ps ux | grep postgres
   90 root      0:00 grep postgres

I've tried to change the host IP address in database URI to 172.19.0.2 which is the IP address of the PostgreSQL server. There isn't any change. It is still refusing to connect.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Mark
  • 1,385
  • 3
  • 16
  • 29

1 Answers1

0
  1. Double check that Postgres is running and listening on that port using ps ux | grep postgres and sudo netstat -plunt | grep postgres

  2. Double check your pg_hba.conf is allowing connections, usually need an entry like the following in addition to having listen_addresses='*' in postgresql.conf

    host all all 0.0.0.0/0 md5

  3. Try manually connecting with psql with the same username and password as your docker-compose.yml file.

Fathom
  • 36
  • 3
  • Something I noticed is that services work just fine when ran locally but in gitlab runner it throws error about not being able to connect to port 5432. I've added outputs to my question. – Mark Mar 28 '23 at 20:54
  • How is gitlab runner running docker? The port 5432 not getting exposed sounds similar to this issue: https://stackoverflow.com/questions/35928670/docker-container-for-postgres-9-1-not-exposing-port-5432-to-host – Fathom Mar 28 '23 at 21:02
  • I've added `network_mode: bridge` to the pipeline config which supposed to place all the services under one network hence making it easier to connect to one another. By far no change. Still connection refused. – Mark Mar 29 '23 at 07:43