How to run knex migrations from Dockerfile or docker-compose

Question

I have Dockerfile that works with API and MySQL database and it should do migrations:

FROM node

WORKDIR /api

COPY . .

RUN npm install

EXPOSE 3001

VOLUME [ "/api/node_modules" ]

CMD [ "npm", "start" ]

Also, there is a docker-compose file where I have database as a service:

  db:
    image: mysql
    container_name: database
    ports:
      - "3306:3306"
    environment:
      MYSQL_ROOT_PASSWORD: password
      MYSQL_PASSWORD: password
      MYSQL_DATABASE: testdb

The problem is, I don't know how to run migrations. Should I do it from docker-compose file or Dockerfile?

I was trying to do something like this in Dockerfile, but it doesn't seem to be working:

...
CMD [ "knex", "migrate:latest" ]
...

Or:

...
RUN knex migrate:latest
...

For a different technology stack, there's some discussion of the options in [How do you perform Django database migrations when using Docker-Compose?](https://stackoverflow.com/questions/33992867/how-do-you-perform-django-database-migrations-when-using-docker-compose). Broadly, the major options are to `docker-compose run your-app knex migrate:latest` manually, or to set up an `ENTRYPOINT` wrapper script to do it automatically on every startup. Does that question have enough detail to help you, even if you're on a different language/migration system? — David Maze, Jan 29 '22 at 22:15
I agree with you. I can create such `ENTRYPOINT` script and this will work, I guess, but I would like to implement it only using docker and docker-compose tools. — dokichan, Jan 29 '22 at 22:31

The Fool · Answer 1 · 2022-01-30T14:06:33.390

Chaining the command or using and entrypoint is less optimal in case you want to horizontally scale your application.

Then all replicas will do the migration, at the same time. It's likely to not cause real problems, but It's still not perfect IMO.

Instead, this should be handled separately, as a one shot command when actually required. For example, in Kubernetes in would be good to run a dedicated migration job along with your application release, if the database schema has actually changed.

With compose, there are no jobs, but you can achieve similar behaviour.

services:
  migration:
    image: busybox
    command: sh -c 'echo "running migration..."; sleep 20; echo "migration completed"'
  app:
    image: busybox
    command: echo "app started"
    depends_on:
      migration:
        condition: service_completed_successfully
    deploy:
      replicas: 3

Now you row the migration only once and all 3 app replicas wait for the migration to complete before they start up.

$ docker compose up
Attaching to app_1, app_2, app_3, migration_1
migration_1  | running migration...
migration_1  | migration completed
migration_1 exited with code 0
app_2        | app started
app_3        | app started
app_1        | app started

In your case, you would use the same image you build from the Dockerfile for both migration and app service. In the migration service you use knex migrate and in the app service you use npm run start.

If you need the migration to even wait for the DB, depends_on might not be sufficient, unless you build in a health check that reflects if the database is actually ready to accept a connection. If you have a health check, then you can use the condition service_healthy.

For example, you could dome something like this.

services:
  db:
    image: mysql
    environment:
      MYSQL_ROOT_PASSWORD: "root"
      MYSQL_DATABASE: "wordpress"
      MYSQL_USER: "wordpressuser"
      MYSQL_PASSWORD: "wordpresspassword"
    healthcheck:
      test: mysqladmin -u root --password=$$MYSQL_ROOT_PASSWORD ping
      interval: 30s
      timeout: 10s
      retries: 10

  migration:
    image: busybox
    command: sh -c 'echo "running migration..."; sleep 20; echo "migration completed"'
    depends_on:
      db:
        condition: service_healthy

  app:
    image: busybox
    command: echo "app started"
    depends_on:
      migration:
        condition: service_completed_successfully
    deploy:
      replicas: 3

You can health check logs by doing a container inspect.

$ docker inspect sample_db_1 --format \
  '{{range .State.Health.Log}}{{.End}} | Exit Code: {{.ExitCode}} | {{.Output}}{{end}}'
2022-01-30 12:53:43.749365 +0000 UTC | Exit Code: 0 | mysqladmin: [Warning] Using a password on the command line interface can be insecure.
mysqld is alive

If you don't want to use a health check, you can also use third party solutions like https://github.com/Eficode/wait-for.

dokichan · Accepted Answer · 2022-01-30T10:50:10.093

0

I solved this problem, probably in stupid way, but it works. So, what I did is just added this on my API container:

restart: on-failure
command: bash -c "npm run knex && npm run start"

Now, it just restarts container until get connection to database and does all migrations.

edited Jan 30 '22 at 10:50

answered Jan 30 '22 at 09:24

dokichan

857
2
11
44

How to run knex migrations from Dockerfile or docker-compose

2 Answers2