How can I get around Docker's limitations on use of symlinks?

Question

I have multiple sets of files I want to include as data for a web server. Some of these sets are the static assets to back a static website.

Suppose there are static assets for exactly two websites, to be included as data for a web server. Each set of assets is laid out in its own folder, perhaps within a source code (e.g. git) repository. These are located at:

/path/to/site1/public

and

/other/path/to/site2/public

In fact there may be several sets of such static assets located in different subdirectories of folders in various places in the filesystem.

I need to install them into a Docker image as, for example, /var/www/http/site1, and /var/www/http/site2, and so on.

It would be very useful to have a good way to bypass the Docker limitations on use of symlinks without having to post these datasets on a web server somewhere and refer to their URLs in Dockerfiles.

Answer 1

This pattern is flexible and has worked well for me. The current directory in this example is the parent of the one containing the static assets, which are in a folder named "public".

In this folder a shell script like this:

#!/bin/bash

# Build a Docker image "site1-assets" with the static files.
docker build -f - --tag=site1-assets . <<EOF
FROM scratch
COPY public ./
EOF

Do similarly for static assets for site2, etc.. It doesn't matter where each set of files is located, as long as they are all on the same machine.

Then in any Dockerfile that need the assets, lines like this:

FROM site1-assets as site1
COPY / /

FROM site2-assets as site2
COPY / /

# and so on.

# And now the step that uses the external files.
FROM alpine

# Some config lines here ...

# Copy in files in image "site1".
COPY from=site1 / /var/www/http/site1/
# Copy in files in image "site2".
COPY from=site2 / /var/www/http/site2/

# More config lines if needed.

CMD run-my-app args

If the external images change rarely, you might wish to copy the files in near the beginning of the client Dockerfile. If they change frequently, it might be better to put them near the end. The idea would be to take advantage of Docker's ability to cache partial image for use in future builds.

The core idea is basically the same as the option "share a base image" in https://stackoverflow.com/a/39382248/5022006, but more flexible than the scenario described there.

Answer 2

Symlinks work just fine inside Docker containers. The limitation with putting symlinks in docker images and containers is that only the link is copied, not the files it points to. Also, inside of a container, the symlink is relative to files and directories inside the image/container filesystem. Therefore, to use symlinks inside of a container, you need to make them relative to a directory structure that will also exist inside of the container. E.g. if you have the following:

path1/projA/link-good -> ../projB/file
path1/projA/link-bad1 -> /host/path1/projB/file
path1/projA/link-bad2 -> ../../path2/projC/file
path1/projA/link-bad3 -> ../projD/file
path1/projB/file
path2/projC/file
path1/projD/file

And you include those in your Dockerfile with:

COPY path1/projA/ /image/projA/
COPY path1/projB/ /image/projB/
COPY path2/projC/ /image/projC/

only the link-good link will work. The other links have the following issues:

• link-bad1 points to a path that only exists on the host, use a relative path instead.
• link-bad2 points to a relative path that doesn't exist inside the image because you added directories as /image/proj*, so there is no path2 inside the image filesystem.
• link-bad3 points to a file that was never included in the image, the COPY command does not follow links.