21

First of all, I want to make it clear I've done due diligence in researching this topic. Very closely related is this SO question, which doesn't really address my confusion.

I understand that when VOLUME is specified in a Dockerfile, this instructs Docker to create an unnamed volume for the duration of the container which is mapped to the specified directory inside of it. For example:

# Dockerfile
VOLUME ["/foo"]

This would create a volume to contain any data stored in /foo inside the container. The volume (when viewed via docker volume ls) would show up as a random jumble of numbers.

Each time you do docker run, this volume is not reused. This is the key point causing confusion here. To me, the goal of a volume is to contain state persistent across all instances of an image (all containers started from it). So basically if I do this, without explicit volume mappings:

#!/usr/bin/env bash
# Run container for the first time
docker run -t foo

# Kill the container and re-run it again. Note that the previous 
# volume would now contain data because services running in `foo`
# would have written data to that volume.
docker container stop foo
docker container rm foo

# Run container a second time
docker run -t foo

I expect the unnamed volume to be reused between the 2 run commands. However, this is not the case. Because I did not explicitly map a volume via the -v option, a new volume is created for each run.

Here's important part number 2: Since I'm required to explicitly specify -v to share persistent state between run commands, why would I ever specify VOLUME in my Dockerfile? Without VOLUME, I can do this (using the previous example):

#!/usr/bin/env bash
# Create a volume for state persistence
docker volume create foo_data

# Run container for the first time
docker run -t -v foo_data:/foo foo

# Kill the container and re-run it again. Note that the previous 
# volume would now contain data because services running in `foo`
# would have written data to that volume.
docker container stop foo
docker container rm foo

# Run container a second time
docker run -t -v foo_data:/foo foo

Now, truly, the second container will have data mounted to /foo that was there from the previous instance. I can do this without VOLUME in my Dockerfile. From the command line, I can turn any directory inside the container into a mount to either a bound directory on the host or a volume in Docker.

So my question is: What is the point of VOLUME when you have to explicitly map named volumes to containers via commands on the host anyway? Either I'm missing something or this is just confusing and obfuscated.

Note that all of my assertions here are based on my observations of how docker behaves, as well as what I've gathered from the documentation.

void.pointer
  • 24,859
  • 31
  • 132
  • 243

1 Answers1

20

Instructions like VOLUME and EXPOSE are a bit anachronistic. Named volumes as we know them today were introduced in Docker 1.9, almost three years ago.

Before Docker 1.9, running a container whose image had one or more VOLUME instructions (or using the --volume option) was the only way to create volumes for data sharing or persistence. In fact, it used to be a best practice to create data-only containers whose sole purpose was to hold one or more volumes, and then share those volumes with your application containers using the --volumes-from option. Here's some articles that describe this outdated pattern.

Also, check out moby/moby#17798 (Data-only containers obsolete with docker 1.9.0?) where the change from data-only containers to named volumes was discussed.

Today, I consider the VOLUME instruction as an advanced tool that should only be used for specialized cases, and after careful thought. For example, the official postgres image declares a VOLUME at /var/lib/postgresql/data. This can improve the performance of postgres containers out of the box by keeping the database data out of the layered filesystem. Docker doesn't have to search through all the layers of the container image for file requests at /var/lib/postgresql/data.

However, the VOLUME instruction does come at a cost.

  • Users might not be aware of the unnamed volumes being created, and continuing to take up storage space on their Docker host after containers are removed.
  • There is no way to remove a volume declared in a Dockerfile. Downstream images cannot add data to paths where volumes exist.

The latter issue results in problems like these.

For the GitLab question, someone wants to extend the GitLab image with pre-configured data for testing purposes, but it's impossible to commit that data in a downstream image because of the VOLUME at /var/opt/gitlab in the parent image.

tl;dr: VOLUME was designed for a world before Docker 1.9. Best to just leave it out.

S-Man
  • 22,521
  • 7
  • 40
  • 63
King Chung Huang
  • 5,026
  • 28
  • 24
  • 3
    This is basically the answer, 'history'. Named volumes didn't use to exist, and `--volumes-from` was prevalent. One other minor addition is documentation, like EXPOSE (which technically isn't needed because you do whatever you want with `run`, it tells the consumer what locations _should_ be a volume / hold persistent data). – johnharris85 Sep 29 '18 at 18:32
  • 1
    Yeah, `EXPOSE` used to play a much larger role with container linking (the `--link` option). I tell developers to just treat it as documentation, today. – King Chung Huang Sep 29 '18 at 18:35
  • 1
    Ah, that explains why to prevent some images from creating superfluous volumes, I have to mount those volumes as `tmpfs`. I can see the issues. Would have been nice if docker deprecated those commands, i.e. a warning diagnostic from `docker build` would be observed if they are used in newer versions of Docker. So I now know going forward to avoid `VOLUME` and `EXPOSE`. I guess, then that leaves one question: How do you describe the "interface" to your image (volumes/ports/etc that should be mapped or otherwise relevant in the host environment or to other containers)? – void.pointer Sep 29 '18 at 22:42
  • Basis for my last question: If anything, `VOLUME` and `EXPOSE` do serve as a sort of documentation. You can easily spot which aspects of the container's environment are relevant to the outside world. – void.pointer Sep 29 '18 at 22:44
  • 2
    I suggest avoiding `VOLUME`, but the `EXPOSE` instruction is useful for documentation. Unlike `VOLUME`, the `EXPOSE` instruction doesn't have any unwanted side effects. – King Chung Huang Sep 30 '18 at 18:55
  • 1
    Since I posted this question, I realized `VOLUME` still serves a functional purpose: It allows you to initialize a volume with data from inside the image. If I do not use `VOLUME` in my `Dockerfile` but still use `-v` when I do `docker run`, will my volume be initialized with the current contents of the directory in the container, if the image copied files to that location when it was built? – void.pointer Mar 04 '19 at 15:36