0

I have a docker-compose project with two containers running NGINX and gunicorn with my django files. I also have a database outside of docker in AWS RDS. My question is similiar to this one. But, that question is related to a database that is within docker-compose. Mine is outside.

So, if I were to open a bash terminal for my container and run py manage.py makemigrations the problem would be that the migration files in the django project, for example: /my-django-project/my-app/migrations/001-xxx.py would get out of sync with the database that stores which migrations has been applied. This will happen since my containers can shutdown and open a new container at any time. And the migration files would not be saved.

My ideas are to either:

  • Use a volume inside docker compose, but since the migrations folder are spread out over all django apps that could be hard to achieve.

  • Handle migrations outside of docker, that would require some kind of "master" project where migration files would be stored. This does not seem like a good idea since then the whole project would be dependent on some locals file existing.

I'm looking for suggestions on a good practice how I can handle migrations.

EDIT:

Here is docker-compose.yml, I'm runing this locally with docker-compose up and in production to AWS ECS with docker compose up. I left out some aws-cloudformation config which should not matter I think.

docker-compose.yml

version: '3'

services:
  web:
    image: <secret>.dkr.ecr.eu-west-3.amazonaws.com/api-v2/django:${IMAGE_TAG}
    build:
      context: .
      dockerfile: ./Dockerfile
    networks: 
      - demoapp
    environment:
      - DEBUG=${DEBUG}
      - SECRET_KEY=${SECRET_KEY}

  nginx:
    image: <secret>.dkr.ecr.eu-west-3.amazonaws.com/api-v2/nginx:${IMAGE_TAG}
    build:
      context: .
      dockerfile: ./nginx.Dockerfile
    ports:
      - 80:80
    depends_on:
      - web
    networks: 
      - demoapp
Felix Eklöf
  • 3,253
  • 2
  • 10
  • 27
  • Can you post your docker-compose file without secrets? Do you have development and production enviroments? – 404pio Apr 27 '21 at 11:01
  • I've add docker-compose.yml – Felix Eklöf Apr 27 '21 at 11:05
  • 1
    normal workflow is: you make changes in models on your dev env -> makemigrations on your dev env -> migrate on your dev env -> commit and push migration files to repo -> make deployment to production -> deployment invoke migrate command. Dev and production enviroments have different databases. – 404pio Apr 27 '21 at 11:06
  • Ahh, I see. I'm not sure how Django migrations works exactly under the hood. If I use this method, let's say I destroy my database and want to migrate it again. Would `py manage.py migrate` work then? – Felix Eklöf Apr 27 '21 at 11:09
  • 1
    If you destroy database, you should create empty one. But I think yes – 404pio Apr 27 '21 at 11:12
  • 1
    Thank you very much, this makes alot of sense! – Felix Eklöf Apr 27 '21 at 11:14
  • 1
    Would be great to describe in separate answer what was exactly the problem and how you solved it. You can answer your question. Maybe your post will be helpfull for other people. – 404pio Apr 27 '21 at 11:38
  • If your database is outside then it does not matter where you run migrations. – Alex Yu Apr 27 '21 at 15:00
  • Maybe not where I run them. But I have to store the migration files somewhere. If I delete them and generate them again and try to run on the database it will tell my they are inconsistent. But I solved by storing them in repo. – Felix Eklöf Apr 28 '21 at 07:09

1 Answers1

0

The problem boiled down to where I would store my migration files that Django generates upon py manage.py makemigrations and when/where I would run py manage.py migrate. As 404pio suggested you can simple store these in your code repo like GitHub.

So my workflow goes like this:

  1. In my local development environment, run py manage.py makemigrations and py manage.py migrations, (target a development database like sqlite).
  2. If everything OK, commit and push to git.
  3. (I'm using CircleCI to test and deploy my Django project, but this could be done manually aswell.) CircleCI runs pipeline after git push. In pipeline I have as the very last step to run py manage.py migrate. This must be after deployment of app since that might fail and then you don't want to migrate.
Felix Eklöf
  • 3,253
  • 2
  • 10
  • 27