0

Summary

With docker-compose, mounting a volume with a destination directory that exists in the Nginx base image, works as expected. However, I get an error with docker-compose up when the mount target is a subdirectory created in the derived image. Here's the gist of the error:

mounting "newdir" to rootfs at "/usr/share/nginx/html/newdir" caused: mkdir newdir: read-only file system: unknown

I don't understand why it seems to be attempting to create the directory (newdir) when it already exists!

Details

Full error below:

ERROR: for test_nginx_1  Cannot start service nginx: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: rootfs_linux.go:76: mounting "/home/user/test/newdir" to rootfs at "/usr/share/nginx/html/newdir"  caused: mkdir /var/lib/docker/overlay2/accdaa147d7825a7d2c71b68f76e0b8663d00f6e46fcfb5c0ec32ada8baf85af/merged/usr/share/nginx/html/newdir: read-only file system: unknown

ERROR: for nginx  Cannot start service nginx: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: rootfs_linux.go:76: mounting "/home/user/test/newdir" to rootfs at "/usr/share/nginx/html/newdir" caused: mkdir /var/lib/docker/overlay2/accdaa147d7825a7d2c71b68f76e0b8663d00f6e46fcfb5c0ec32ada8baf85af/merged/usr/share/nginx/html/newdir: read-only file system: unknown.

Here is the docker-compose.yml:

version: '3'
services:
  nginx:
    image: local/mynginx:withsubdir
    volumes:
      - ./html:/usr/share/nginx/html:ro
      - ./newdir:/usr/share/nginx/html/newdir

The above image, local/mynginx:withsubdir, was built with the following Dockerfile:

FROM docker.io/library/nginx:1.21
RUN  mkdir /usr/share/nginx/html/newdir && ls -lah /usr/share/nginx/html/newdir

And I verified that I can mount the volume and see its content with docker:

$ docker run --rm -ti -v /home/user/test/newdir:/usr/share/nginx/html/newdir local/mynginx:withsubdir bash
root@7db709476271:/# ls /usr/share/nginx/html/newdir
hallo.imhere

So the issue seems specific to docker-compose. If I remove the last line in docker-compose.yml then docker-compose up starts successfully. How can I mount the directory created in the derived image? Thank you.

Extra Notes

By the way, my original docker-compose.yml is longer and it has a build step that points to the Dockerfile that creates that new directory. But I created this smaller reproducer while troubleshooting, and I've built the new image manually with:

docker build . -t local/mynginx:withsubdir

I get the same error in two different environments (Docker version 20.10.7, build 20.10.7-0ubuntu1~20.04.1 and Docker version 20.10.12, build e91ed5707e), with or without :ro.

My test directory files:

.
├── docker-compose.yml
├── Dockerfile
├── html
│   └── index.html
└── newdir
    └── hallo.imhere

I did some online research but could not find any solution.

PR7
  • 1,524
  • 1
  • 13
  • 24
Nagev
  • 10,835
  • 4
  • 58
  • 69
  • Does the directory `./html/newdir` exist on your host system? (It can be empty, but does it exist at all?) – David Maze Apr 17 '22 at 01:31
  • No, the host tree is as shown. The destination path does not have to exist in the host. If I leave only the second mount `./newdir:/usr/share/nginx/html/newdir` it works fine, the issue was the overlapping mounts, as explained in my answer. – Nagev Apr 17 '22 at 12:16
  • the reason is, if your file does not exist docker creates it as a directory causing this weird error – Dr Deo Jul 09 '22 at 11:54

2 Answers2

4

Docker is trying to create the mount point in the read-only directory, and failing.

The normal Linux mount(8) call, and the underlying mount(2) system call, mount some filesystem on some existing directory. A Linux bind mount is a special case of this where the filesystem that's being mounted is also an existing directory, and a Docker bind mount uses this same mechanism.

In general, in Docker, if the target of any kind of mount doesn't already exist, Docker will create it as a new empty directory. Normally this is in the container filesystem, but if you're trying to mount a volume inside some other kind of mounted volume, the mount point will be created in the outer volume.

One easy way to demonstrate this is with a plain docker run command, where you mount an outer and an inner volume, but the outer volume is actually a bind mount:

# Create a new empty local "outer" directory
rm -rf outer
mkdir outer

# Create a new empty container...
docker run --rm \
  -v "$PWD/outer:/outer" \ # bind-mounting the outer directory
  -v inner:/outer/inner  \ # mounting a named volume within that
  busybox                \ # running any image
  ls /outer                # and a command that exits immediately

# Look inside the host directory, and an empty `inner` directory will exist
ls ./outer

# Clean up
docker volume rm inner
rm -rf outer

(This is the same mechanism as Why does docker create empty node_modules and how to avoid it?.)

Now, with these mechanics in mind, you've told Docker to mount the outer directory as read-only, and it does. Then the newdir directory doesn't exist in the container filesystem, so Docker tries to create it as the mount point, but it can't, because the directory is in a read-only filesystem. That's the error you get.

With pure Docker volumes, some possible workarounds are to:

  1. Create the mount point on your host filesystem before you start the container

    mkdir ./html/newdir
docker-compose up -d

  2. Don't mount the outer directory as read-only

    volumes:
  - ./html:/usr/share/nginx/html  # not ...:ro

  3. Mount the inner directory somewhere else, as you've done in your answer

    volumes:
  - ./html:/usr/share/nginx/html:ro
  - ./newdir:/usr/share/nginx/newdir:ro  # not under .../html

A better answer still could be to avoid volumes here entirely. In a Dockerfile you can COPY this content into your image. Since the container filesystem is isolated from the host, this removes the risk of the container modifying the host files, and you can restructure the content however you want within the Dockerfile. This also means you can run the image on another system without also having a copy of the files you want to serve there.

FROM nginx
COPY html/ /usr/share/nginx/html/
COPY newdir/ /usr/share/nginx/html/newdir/
# no need to repeat base image's EXPOSE or CMD
David Maze
  • 130,717
  • 29
  • 175
  • 215
  • Thank you for the comprehensive answer! I like solution 2 the best. Using volumes instead of `COPY` was *intentional* in my case as it's more convenient during development, but it's a good point. – Nagev Apr 17 '22 at 15:21
1

The issue seems to be the fact that the second destination mount is a subdirectory of the first. I changed my docker-compose.yml to this after building a new image and the error is now gone:

version: '3'
services:
  nginx:
    image: local/mynginx:withsubdir
    volumes:
      - ./html:/usr/share/nginx/html:ro
      - ./newdir:/usr/share/nginx/newdir:ro

Hopefully that's something I can work with, but at least the mystery appears to be solved.

Nagev
  • 10,835
  • 4
  • 58
  • 69