Docker-compose stop removes the data from tables in the database

Question

Everytime I do "docker-compose restart", my initial entered data in the database is automatically removed. I see the data being added to the table but as soon as I do restart or stop the docker and start it again, it doesn't exist anymore. The empty table does exist, It just the data that is deleted. Any help would be greatly appreciated.

docker-compose.yml

version: '3'

services:
    app:
        build: .
        command: python app/manage.py runserver 0.0.0.0:8000
        volumes:
            - ./:/app/
        ports:
            - 8000:8000
        env_file:
            - ./.env.dev
        depends_on:
            - db

    db:
        image: postgres
        volumes:
            - postgres_data:/var/lib/postgresql/data/
        environment:
            - POSTGRES_USER=cmpt_user
            - POSTGRES_PASSWORD=password
            - POSTGRES_DB=cmpt_project 
volumes:
    postgres_data:

Dockerfile

# pull official base image
FROM python:3.8.0-alpine
RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev

# set work directory
WORKDIR /app

# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# install dependencies
RUN pip install --upgrade pip
COPY ./requirements.txt /app/requirements.txt
RUN pip install -r requirements.txt

# copy entrypoint.sh
COPY ./entrypoint.sh /app/entrypoint.sh

# copy project
COPY . /app/

# run entrypoint.sh
ENTRYPOINT ["/app/entrypoint.sh"]

Answer 1

The docker-compose.yml looks good to me:

• You set up a persistent volume which is stored by default at /var/lib/docker/volumes/<project>_postgres_data
• by default <project> is the name of the folder the docker-compose.yml resides
• The volume will survive (I verified) a docker-compose down or docker-compose restart. See https://docs.docker.com/compose/reference/restart/

Not commiting database transaction

Maybe the python program opens a database transaction, insert data to the table(s), but does not commit the transaction therafter. See more about transaction management, auto-commit mode, rollback etc.

Would you mind to provide more information how you verified that your data really was in the tables?

Named volume interferences (MacOS)

Maybe the named volume interferes with the anonymous volume created by the postgres image you are using. Please modify your docker-compose.yaml named volume and environment:

version: '3'

services:
    app:
        build: .
        command: python app/manage.py runserver 0.0.0.0:8000
        volumes:
            - ./:/app/
        ports:
            - 8000:8000
        env_file:
            - ./.env.dev
        depends_on:
            - db
    db:
        image: postgres
        volumes:
            - postgres_data:/var/postgres_data  #<----- modify
        environment:
            - PGDATA: /var/postgres_data        #<----- add
            - POSTGRES_USER=cmpt_user
            - POSTGRES_PASSWORD=password
            - POSTGRES_DB=cmpt_project 
volumes:
    postgres_data:

Source: https://github.com/docker/compose/issues/5012

Answer 2

You seem to have misunderstood how volumes work. At the bottom you have:

volumes:
    postgres_data:

When you do docker-compose up, it will do an implicit docker volume create and create the postgres_data volume. If you do a docker volume ls, you might see something like:

$ docker volume ls | grep postgres_data
local               r_postgres_data

Then, if you look in docker volume inspect r_postgres_data, you'll see:

$ docker volume inspect r_postgres_data
[
    {
        "CreatedAt": "2020-02-07T19:43:11Z",
        "Driver": "local",
        "Labels": {
            "com.docker.compose.project": "r",
            "com.docker.compose.version": "1.24.1",
            "com.docker.compose.volume": "postgres_data"
        },
        "Mountpoint": "/var/lib/docker/volumes/r_postgres_data/_data",
        "Name": "r_postgres_data",
        "Options": null,
        "Scope": "local"
    }
]

I won't get into all the details here (you may want to consult the Docker Community on this). Basically, a docker-compose restart wipes that volume and starts over.

All this to say that based on what you wrote in your original post, you're probably just looking to use a local folder, not a docker-created volume. In this case, you need to add ./ to your path:

...
    db:
        image: postgres
        volumes:
            - ./postgres_data:/var/lib/postgresql/data/
        environment:
            - POSTGRES_USER=cmpt_user
            - POSTGRES_PASSWORD=password
            - POSTGRES_DB=cmpt_project 
...