docker-compose not starting postgres and gives error

Question

I'm really new to Docker (also postgres) and still finding my feet. I get an error and can't seem to get one of my postgres services running, although when I start it, I'm able to access pgadmin and airflow via the browser. I think there is some sort of conflict happening but I'm not sure where. I have a docker-compose.yml file that starts a few containers, as well as the postgres one in question which has the servce name db:

version: '3.7'
services:
    postgres:
        image: postgres:9.6
        environment:
            - POSTGRES_USER=airflow
            - POSTGRES_PASSWORD=airflow
            - POSTGRES_DB=airflow
        logging:
            options:
                max-size: 10m
                max-file: "3"

    db:
        image: postgres:13.0-alpine
        restart: always
        environment:
            POSTGRES_DB: postgres
            POSTGRES_USER: admin_user
            POSTGRES_PASSWORD: secret_password
            # PGDATA: /var/lib/postgresql/data
        volumes:
            - ./db-data:/var/lib/postgresql/data
        ports:
            - "5433:5432"
         
    pgadmin:
        image: dpage/pgadmin4:4.27
        restart: always
        environment:
            PGADMIN_DEFAULT_EMAIL: admin_user@test_email.com
            PGADMIN_DEFAULT_PASSWORD: test_password
            PGADMIN_LISTEN_PORT: 1111
        ports:
        - "1111:1111"
        volumes:
        - pgadmin-data:/var/lib/pgadmin
        links:
        - "db:pgsql-server"

    webserver:
        image: l/custom_airflow:1.5
        container_name: l_custom_airflow
        restart: always
        depends_on:
            - postgres
        environment:
            - LOAD_EX=n
            - EXECUTOR=Local
        logging:
            options:
                max-size: 10m
                max-file: "3"
        volumes:
            - ./dags:/usr/local/airflow/dags
            - ./db-data:/usr/local/airflow/db-data
            - ./pgadmin-data:/usr/local/airflow/pgadmin-data
        ports:
            - "8080:8080"
        command: webserver
        healthcheck:
            test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
            interval: 30s
            timeout: 30s
            retries: 3

volumes:
    db-data: 
    pgadmin-data:

The relevant part is this:

db:
    image: postgres:13.0-alpine
    restart: always
    environment:
        POSTGRES_DB: postgres
        POSTGRES_USER: admin_user
        POSTGRES_PASSWORD: secret_password
        # PGDATA: /var/lib/postgresql/data
    volumes:
        - ./db-data:/var/lib/postgresql/data
    ports:
        - "5433:5432"

[I already have two version of postgres on my local machine, and I saw that they use ports 5432 and then 5433, so it looks like the latest one goes to 5433. Similarly, I have another service (airflow) that depends on an older version of postgres to run, so I assume since that one comes first it takes 5432, and then the new postgres service I want will likely be mapped to 5433 as default - please correct me if I'm wrong]

But when I run docker-compose up -d and check my containers with docker container ls -a I see that this particular container is continuously restarting. I ran docker logs --tail 50 --follow --timestamps pipeline_5_db_1 (the container name for the db service) and I see the following error:

2020-10-28T08:46:29.730973000Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:30.468640800Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:31.048144200Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:31.803571400Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:32.957604600Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:34.885928500Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:38.479922200Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:45.384436400Z chmod: /var/lib/postgresql/data: Operation not permitted                                                                                                                                    2020-10-28T08:46:58.612202300Z chmod: /var/lib/postgresql/data: Operation not permitted

I googled the error and saw a couple of other SO posts but I can't see a clear explanation. This post and this post are a bit unclear to me (might be because I'm not so familiar), so I'm not sure how to use the responses to solve this issue.

Do you have a `db-data` directory in the directory where you run `docker-compose up`? What's the mode and owner of the directory and its contents (output of `ls -la db-data`)? — Carl, Oct 28 '20 at 11:16
Yes, the directory exists there. The output of `ls -la db-data` is `total 0 drwxrwxrwx 1 root root 512 Oct 28 10:12 . drwxrwxrwx 1 root root 512 Oct 28 10:15 .. ` — fffrost, Oct 28 '20 at 11:21
If a bind mounted directory doesn't already exist, docker will create it but it will be owned by root which is often inconvenient and sometimes a major problem. Creating the directory in advance with a different owner can help but it really depends on the images being used. The postgres one apparently does a chmod and then tries to write files as user id 70 but others behave differently. — Carl, Oct 28 '20 at 11:49

Carl · Accepted Answer · 2020-10-28T11:13:44.670

1

You've got dbdata defined as a named volume at the bottom of the compose file but you're using ./dbdata within each service which is a bind mount. You might try using the named volume instead of the shared directory in your db and webserver services, like this:

    volumes:
        - db-data:/var/lib/postgresql/data

A bind mount should also work but can be troublesome if permissions on the mounted directory aren't quite right, which might be your problem.

The above also applies to pgadmin-data where the pgadmin service is using a named volume but webserver is using the bind mount (local directory). In fact, it's not clear why the webserver would need access to those data directories. Typically, a webserver would connect to the database via port 5432 (which doesn't even need to be mapped on the host). See for instance the bitnami/airflow docs on Docker Hub.

edited Oct 28 '20 at 11:13

answered Oct 28 '20 at 10:54

Carl

898
9
16

Thanks, this seems to have done the job. Since I'm editing a template docker-compose.yml (from https://github.com/puckel/docker-airflow), I did as you suggested in my `db` postgres service and its status is now saying Up! – fffrost Oct 28 '20 at 11:20
I assumed that the webserver needed access to it so that its data would persist between starting and stopping the containers, and I wasn't aware that there was a difference between using `./directory_name` and `directory_name`. The permissions thing might come because I'm using WSL2... – fffrost Oct 28 '20 at 11:24
1

Glad it's working. WSL2 may explain the issue with permissions, not something I've tried out yet. Sharing files between containers is sometimes necessary but in many cases it's preferable to have them communicate via the network, and things like databases, webservers, filestores such as Amazon S3, key-value stores like redis, etc are all designed with this in mind. – Carl Oct 28 '20 at 11:54

docker-compose not starting postgres and gives error

1 Answers1