dependency container accesses files from dependent container via volume

Question

Docker compose:

version: "3"
services:

  db:
    container_name: db
    image: mysql
    ports:
      - "3306:3306"
    volumes:
      - initdb:/docker-entrypoint-initdb.d/:ro
  
  web:
    container_name: web
    image: my_web
    volumes:
      - initdb:/initdb/:ro
    depends_on: 
      - db

volumes:
  initdb:

my_web image contains /initdb/create_schema.sql.

web depends on db. So the service db will start before web. Will the create_schema.sql from web be available for the db to consume?

From testing, it is available. Is this guaranteed or by random?

The create_schema.sql is generated when building my_web image. It is not on host disk.

Trying to understand when volumes are created and mapped to containers. If they are created and mapped before starting all containers, then the shared data via volumes will be guaranteed to be available.

score 1 · Accepted Answer · answered Apr 15 '23 at 11:01

The approach as you've described it will not work reliably. Most application frameworks include some sort of database migration system, and a more robust approach will be to have your application run migrations on startup than to try to use a named volume like this.

Docker named volumes have a subtly tricky sequence for copying content into a volume. When a container is created that mounts the volume, if and only if the volume is totally empty, then content is copied from the mount point into the image. This only happens if the volume is empty (so it will never see changes to the image content), and it only happens for Docker named volumes and not other kinds of mounts (Docker bind mounts, Kubernetes PersistentVolumeClaims, ...).

What this sequence means is that Compose will

Create the named volume
Create and start the db container, mounting the empty volume
Create and start the web container, at which point Docker will copy the script into the volume

This leads to a race condition where it's not certain whether the web container startup will copy the file first, or the db container starts looking through that directory for initialization scripts.

If you extend this setup to use a health check to wait until the database is running, then this won't work at all. The web container won't be created until the db container has finished its initialization, which means you have a guarantee that the volume will be empty when the /docker-entrypoint-initdb.d directory is scanned.

Again, remember that Docker won't ever update the contents of the volume, so if you change the initialization script, the volume will still have the old one. The standard database images also only run /docker-entrypoint-initdb.dif their data directory is uninitialized. Running a migration system avoids both of these problems.

Is there a way to copy generated files(e.g., create_schema.sql) out to host when building the my_web image? Thanks. — eastwater, Apr 15 '23 at 19:34
No (but it's likely part of your source tree). Even ignoring volumes, I would still generally prefer a migration system to a one-time database-initialization script. — David Maze, Apr 15 '23 at 20:06

score 0 · Answer 2 · answered Apr 15 '23 at 09:47

if you want

The create_schema.sql is generated when building my_web image. It is not on host disk.

than you need to create third container (it can be generated with similar Dockerfile to file for web but with different entrypoint) lets name it init-db-script, start it before db container, generate create_schema.sql in it

then add it in depends for db

  db:
    depends_on: 
      - init-db-script
    ...

Or you can try to create data in volume by docker run for 'script container' that will stops afer scripts compleat, but data in volume will be created.

But I suggest to generate create_schema.sql before build script, add it to your repository and add copy of it to Docker image of db in its Dockerfile

dependency container accesses files from dependent container via volume

2 Answers2