Docker-compose named mounted volume

Question

In order to keep track of the volumes used by docker-compose, I'd like to use named volumes. This works great for 'normal' volumes like

version: 2
services: 
  example-app:
    volume:
      -named_vol:/dir/in/container/volume
volumes:
  named_vol:

But I can't figure out how to make it work when mounting the local host. I'm looking for something like:

version: 2
services: 
  example-app:
    volume:
      -named_homedir:/dir/in/container/volume
volumes:
  named_homedir: /c/Users/

or

version: 2
services: 
  example-app:
    volume:
      -/c/Users/:/home/dir/in/container/ --name named_homedir

is this in any way possible or am I stuck with anonymous volumes for mounted ones?

Did you find the answer? I'm having a similar problem, but perhaps I don't understand what volumes are supposed to be: http://stackoverflow.com/questions/41334488/error-in-file-docker-compose-yml-volume-must-be-a-mapping-not-a-string/41334607#41334607 — Richard, Dec 29 '16 at 11:15
i didnt really, not withing compose, still using 'services' with 'empty' container and a volume (then use volumes_from that container) — pvgoddijn, Dec 29 '16 at 11:18

score 67 · Accepted Answer · edited Dec 08 '20 at 21:09

67

As you can read in this GitHub issue, mounting named volumes now is a thing … since 1.11 or 1.12.). Driver specific options are documented. Some notes from the GitHub thread:

docker volume create --opt type=none --opt device=<host path> --opt o=bind

If the host path does not exist, it will not be created.

Options are passed in literally to the mount syscall. We may add special cases for certain "types" because they are awkward to use... like the nfs example [referenced above].

^{– @cpuguy83}

To address your specific question about how to use that in compose, you write under your volumes section:

my-named-volume:
     driver_opts:
           type: none
           device: /home/full/path #NOTE needs full path (~ doesn't work)
           o: bind

This is because as cpuguy83 wrote in the github thread linked, the options are (under the hood) passed directly to the mount command.

EDIT: As commented by…

…@villasv, you can use ${PWD} for relative paths.

…@mikeyjk, you might need to delete preexisting volumes:

 docker volume rm $(docker volume ls -q)
 OR
 docker volume prune

…@Camron Hudson, in case you have no such file or directory errors showing up, you might want to read this SO question/ answer as Docker does not follow symlinks and there might be permission issues with your local file system.

edited Dec 08 '20 at 21:09

roxch

351
3
16

answered Oct 13 '16 at 20:38

kaiser

21,817
17
90
110

4

How can the compose file in the original question take advantage of that? – Dave L. Nov 15 '16 at 17:50
1

Create the volume like in the above example, and then set the volume as external on the docker-compose file https://docs.docker.com/compose/compose-file/#/external. – eduncan Jan 24 '17 at 03:18
1

@david you can have in your composer file the following: (shit formatting..) my-named-volume: \n driver_opts: \n type: none \n device: /home/user/your/path/ #NOTE needs full path \n o: bind \n – linuxbandit Apr 19 '17 at 12:49
1

@linuxbandit You can use backticks for inline code formatting (see the _help_ on the right hand side of this textarea). Or: Just add an [edit] to the answer. It's appreciated :) – kaiser Apr 19 '17 at 20:20
Thanks @Kaiser, I did so! Also I didn't know the formatting worked also in these comments (I mean, the indentation and such) – linuxbandit Apr 21 '17 at 09:55
2

You can avoid the use of complete full paths using `${PWD}` to make your project more portable. – villasv Feb 07 '18 at 20:27
1

I had to delete preexisting volumes with ~'docker volume rm $(docker volume ls)', but it is now working :). – mikey Jun 12 '18 at 01:42
1

Here is a `${PWD}`, present working directory, example: `.env` contains: `IPFS_STAGING="staging-dir"` Sample `docker-compose.yml` usage `- "${PWD}${IPFS_STAGING}:/export:rw"` – njappboy Aug 27 '18 at 00:25
@njappboy And this only works with uppercase `${PWD}`? Is this documented somewhere and with which version is this available? – kaiser Aug 27 '18 at 05:12
@kaiser `$PWD` is a environment variable typically set. `pwd` is slightly different. https://unix.stackexchange.com/questions/174990/what-is-pwd-vs-current-working-directory#175000 – njappboy Sep 01 '18 at 01:42
This does not work for me. I get `error while mounting volume with options: type='none' device='/home/full/path' o='bind': no such file or directory`. – Cameron Hudson Jun 10 '19 at 18:01
@CameronHudson Maybe double check the Docker Compose version you are using. – kaiser Jun 11 '19 at 13:07
1

@kaiser I discovered that the `no such file or directory` error was wrong, and that the real problem was that I needed to grant Docker permission to access the path with Docker Desktop. More info here: https://stackoverflow.com/questions/56531493/how-can-i-mount-an-absolute-host-path-as-a-named-volume-in-docker-compose – Cameron Hudson Jun 12 '19 at 15:22

villasv · Answer 2 · 2018-02-16T15:31:23.957

19

OP appears to be using full paths already, but if like most people you're interested in mounting a project folder inside the container this might help.

This is how to do it with driver_opts like @kaiser said and @linuxbandit exemplified. But you can try to use the usually available environment variable $PWD to avoid specifying full paths for directories in the docker-compose context:

logs-directory:
  driver_opts:
    type: none
    device: ${PWD}/logs
    o: bind

edited Feb 16 '18 at 15:31

answered Feb 08 '18 at 14:53

villasv

6,304
2
44
78

Is it possible to mount just one file via named bind? – Eugen Konkov Oct 13 '22 at 06:55

score 12 · Answer 3 · edited Jul 17 '20 at 02:44

12

I've been trying the (almost) same thing and it seems to work with something like:

version: '2'
services: 
  example-app:
    volume:
      -named_vol:/dir/in/container/volume
      -/c/Users/:/dir/in/container/volume
volumes:
  named_vol:

Seems to work for me (I didn't dig into it, just tested it).

edited Jul 17 '20 at 02:44

Pang

9,564
146
81
122

answered Jul 11 '17 at 16:07

rmNyro

351
2
14

8

deleting this line `- named_vol:/dir/in/container/volume` won't affect your config. It''ll still work. Your creating a named volume called `named_vol` and then basically mounting it, and then mounting something else in it's place, so now, `named_vol` isn't mounted on the container any more. To confirm this, you can inspect the container and check if the volume called `named_vol` is mounted to the container. – kapad Nov 21 '17 at 09:26
1

As the question suggests, @pvgoddijn wanted to have a named volume (so deleting the line will surely affect that) but also mount host volume into it. So basically it's a bit hacky but what it does is mount a volume to the container folder then mount the host folder to the very same place. That way you can use the name of the volume and also edit the container files directly from the host. Though it was more of an experiment for me and I wouldn't really recommend that. I think you need to redesign your infra if your forced to do such a thing. – rmNyro Nov 21 '17 at 16:30
As far as I can tell, this won't give you one of the best benefits of using a named volume: container data being copied to host, when mounting an empty dir. So as @kapad states, you may as well just use a 'bind mount'. Unless there is some other benefit I'm not aware of. – mikey Jun 11 '18 at 13:33
Hu, correct me if I'm wrong but, named volume are not copied to the disk, the "name" is like an "id". So you can "call" the volume by its id from another container. Data are "bound" (copied) to the host on the second line. But then again, I didn't dig into it as stated, I was just playing with docker for my work at the time and thought it would be a good answer to this question (maybe the question is wrong ^^) that's all. – rmNyro Jun 13 '18 at 08:28
Named volumes reside on the host disk inherently. When you mount an empty named volume, the corresponding container contents are copied to the host (see: https://docs.docker.com/storage/volumes). The first line creates a named volume. The second line creates a bind mount, which will always overwrite the containers files with the host contents (which is empty). This solution would still need you to copy the container contents manually to host to access from /c/Users/. I think the question is a good one and I think @kaiser has the best answer (based on cpuguy83's solution from Github). – mikey Jun 14 '18 at 06:34
Good for him, that option (as stated) didn't exist at the time. Regarding named vol vs host mounted vol you're right, both are on the host disk. Only anonymous vol stays in ram as I understand it. That being said, my answer was somewhat "hacky" and does not prevent new and better one so... :) – rmNyro Jun 21 '18 at 08:37

score 7 · Answer 4 · answered Apr 28 '16 at 08:24

7

I was looking for an answer to the same question recently and stumbled on this plugin: https://github.com/CWSpear/local-persist Looks like it allows just what topic started wants to do.

Haven't tried it myself yet, but thought it might be useful for somebody.

answered Apr 28 '16 at 08:24

Stas Makarov

716
7
15

1

Looks interesting but for the sake of easy deployment, i think i'll probably wait out till something like this ends up in docker(-compose) itself – pvgoddijn Apr 28 '16 at 10:59
What you are saying is analogous to saying you wont download an app from an app store, but rather expect it to be built into the OS someday. Not going to happen, it seems: _@CWSpear We won't support this in the built-in driver. You are more than welcome to build a plugin to handle this._ https://github.com/docker/docker/issues/19990 – Peeter Kokk Jul 12 '16 at 12:44
2

For an example of using local-persist in docker-compose.yml, see here: https://github.com/CWSpear/local-persist/blob/master/docker-compose.example.yml – Peeter Kokk Jul 12 '16 at 13:24

score 6 · Answer 5 · answered Mar 09 '16 at 17:25

6

Host volumes are different from named volumes or anonymous volumes. Their "name" is the path on the host.

There is no way to use the volumes section for host volumes.

answered Mar 09 '16 at 17:25

dnephin

25,944
9
55
45

1

Host volumes here is presumably referring to bind mounts. Named volumes 'name' is not the path on the host. The host path can be specified for named volumes as per @cpuguy83's solution: https://github.com/moby/moby/issues/19990#issuecomment-248955005. – mikey Jun 12 '18 at 05:28

Docker-compose named mounted volume

5 Answers5

Linked