175

I'm using docker-compose and v3. I'm trying to mount a volume in docker:

./appdata:/appdata

I'd like to have this as a volume and then reference that volume from multiple containers. The volume configuration reference only shows data-volume: as a named volume, with no value, so it doesn't look like the above.

services:

    nginx:
        build: ./nginx/
        ports:
            - 80:80
        links:
            - php
        volumes:
            - app-volume

    php:
        build: ./php/
        expose:
            - 9000
        volumes:
            - app-volume

volumes:
     app-volume: ./appdata:/appdata

This gives me:

ERROR: In file './docker-compose.yml', volume 'app-volume' must be a mapping not a string.

Obviously I know I need to change the volumes key/value pair, but I'm not sure how to change this so I can share a volume between services.

I've also checked out volumes_from but this effectively just allows inheritance from other containers. I've seen someone use volumes_from on another container that contains the mapping they want, but with command: true set so that the container is never actually run, which to me just seems like a hack.

How can I do this?


Note, I do have the following working:

nginx:
    volumes:
        - ./appdata:/appdata
php:
    volumes:
        - ./appdata:/appdata

But that's just duplication and is something I'm hoping a named volume could help me avoid :-)

Pang
  • 9,564
  • 146
  • 81
  • 122
Jimbo
  • 25,790
  • 15
  • 86
  • 131

4 Answers4

208

The named volumes can be shared across containers in the following way:

services:
    nginx:
        build: ./nginx/
        ports:
            - 80:80
        links:
            - php
        volumes:
            - app-volume:location_in_the_container

    php:
        build: ./php/
        expose:
            - 9000
        volumes:
            - app-volume:location_in_the_container

volumes:
     app-volume: 

Here's an example config that I use for better understanding. I'm exposing the static files generated from my web container to a named volume called static-content which is then read and served by the nginx container:

services:
  nginx:
    container_name: nginx
    build: ./nginx/

    volumes:
      - static-content:/usr/src/app

  web:
    container_name: web
    env_file: .env
    volumes:
      - static-content:/usr/src/app/public
    environment:
      - NODE_ENV=production

    command: npm run package

volumes:
  static-content:
m13r
  • 2,458
  • 2
  • 29
  • 39
Jayaram
  • 6,276
  • 12
  • 42
  • 78
  • 1
    But there are two locations I use in my example, a host one and a container one. Originally it looked like this: `- ./appdata:/appdata` but with your solution, I can only see one of these values? – Jimbo May 31 '17 at 12:35
  • Both the containers can write to the volume AFAIK. so if you expose a folder from one of your containers into the named volume - it can be seen and overwritten by the other container and vice versa. If its still unclear - i'll paste my example code to try explaining it better – Jayaram May 31 '17 at 12:37
  • 129
    Where do you set the location of `static_content` on the host filesystem? – Travis Bear Mar 13 '18 at 22:37
  • 11
    Whitespace in `app-volume: location_in_the_container` is wrong. – hasufell Oct 22 '18 at 03:36
  • 9
    What if `/usr/src/app` in the `nginx` container and `/usr/src/app/public` in the `web` container both got original content, which one will be used and why? – deko Jan 03 '19 at 17:09
  • you're right. The problem is that i completely lost context of this answer as its been quite a while. Regarding your question - its a named volume - i believe the mapping from host-> docker will be done internally by docker @TravisBear - i belive the location (mapping to a local drive) is managed by docker internally. regarding whose content is used, i believe both can write to the volume - so its upto you to ensure theres no overwriting by files. In my case , only node writes to the volume while nginx serves the file – Jayaram Feb 07 '19 at 10:23
  • Thanks for this answer, yet even having pretty much the same docker-compose.yaml as you, I get a `Cannot create container for service mobile-app: invalid volume specification: '..._appdata::rw': invalid mount config for type "volume": field Target must not be empty` I have checked the official [documentation]("https://docs.docker.com/compose/compose-file/#volume-configuration-reference") and there doesn't seen to be a difference. – ilomax Mar 06 '19 at 17:03
  • 3
    @TravisBear for this use case (sharing data between containers) there is no really need to have it on the host. Example with static data is great - you execute `collectstatic` in one container and want results to be available in another one, but you don't care about host folder – The Godfather Mar 20 '19 at 09:45
  • 16
    @Kannaj TravisBear's question is the one that correctly identifies the issue I find the most confusing. How in the compose file can you specify where the named volume is sourced? I don't want to leave it up to the docker engine to determine where to store the named volume on the host, I want to specify a path. – Ben Collins Apr 03 '19 at 20:15
  • @BenCollins - I believe you can use `path based volumes` for your use case - https://nickjanetakis.com/blog/docker-tip-28-named-volumes-vs-path-based-volumes – Jayaram May 03 '19 at 08:56
  • I believe that "path based volumes" is actually referring to "bind mounts" - which are much more easy to google for. – Jimbo May 21 '20 at 21:27
  • 1
    @TravisBear Looks like the new answer from Andriy Ivaneyko below might do this. – Jimbo Jul 14 '20 at 08:07
  • 1
    Does this work for `3+` docker compose schema? For me, directory is always empty, even though another container has data in it. – Andrius Dec 02 '21 at 14:33
47

This solves it without using named volumes:

      volumes:
          - ./appdata:/appdata

So, it looks like:

services:

  nginx:
      build: ./nginx/
      ports:
          - 80:80
      links:
          - php
      volumes:
          - ./appdata:/appdata

  php:
      build: ./php/
      expose:
          - 9000
      volumes:
          - ./appdata:/appdata
Pang
  • 9,564
  • 146
  • 81
  • 122
Robert
  • 33,429
  • 8
  • 90
  • 94
  • 6
    Ah, nice timing! I did this above (see my change). However, it seems we're still duplicating the mapping. If I use this over 3 containers, it gets big. Can we use named containers to avoid this duplication? – Jimbo May 31 '17 at 12:31
  • The thing is that named volumes is not something only about syntax and clear code. It will create a volume inside docker data installation directory and you will not have your local files there (the ./appdata). Is it useful for you anyway? – Robert May 31 '17 at 12:35
  • 3
    I definitely need the `./appdata`, that is what I'm trying to do. Leave this answer here though :) +1 – Jimbo May 31 '17 at 12:53
  • 5
    What happens if I have two containers of the same image, uploading file (via upload file service) in one container, it's gonna be available in the other one? if not, how can I do that? – magnoz Nov 14 '18 at 12:24
32

You can use one of the two options:

  1. Named volumes: https://docs.docker.com/compose/compose-file/07-volumes/

  2. extension-fields to avoid duplicating volumes source and prevent yourself from future typos:

version: '3.5'

x-services-volume:
  &services-volume
  type: bind
  source: ./appdata
  target: /appdata

services:

    nginx:
        build: ./nginx/
        ports:
            - 80:80
        links:
            - php
        volumes: *services-volume

    php:
        build: ./php/
        expose:
            - 9000
        # Use same way as for nginx if target override not needed.
        volumes:
            - <<: *services-volume
            target: /opt/target-override

NOTE: This feature is available starting from version 3.4 file format.

Andriy Ivaneyko
  • 20,639
  • 6
  • 60
  • 82
  • If the *services-volume is just a pointer to the value set above, this looks awesome... I'll have to try it. – Jimbo Jul 13 '20 at 14:51
  • @Jimbo yes, it is, also note that docker-compose file version shall be 3.4+ – Andriy Ivaneyko Jul 13 '20 at 14:54
  • 6
    Named volumes, aka the top-level `volumes` field, seem to still be a thing in [v3 of `docker-compose`](https://docs.docker.com/compose/compose-file/#volumes). – Alex Povel Jul 21 '20 at 16:48
  • I tried exactly this approach to share data between three containers and a host. In my case, the folder structures in the containers are equal. Seems as if this works as expected. What I don't get is, which container (or the host) has the privileged right when starting it up again? What I mean: When issuing a docker-compose down, each container's internal file-system is saved somewhere internally. When I now change data on the host-folder and do a docker-compose up again: What is the rule for the content in the shared folder? Will data from the host overwrite the internally saved data?thx! – and0r Oct 21 '20 at 08:27
  • @and0r it's retained and same for all containers, so you will have content retained. – Andriy Ivaneyko Oct 21 '20 at 09:23
  • @AndriyIvaneyko but thats exactly my question: what are the retaining rules. Again: If I have a shared volume and I delete a file on the host while docker is not running and then I start it up again with the docker compose file: will the file be deleted for all containers or will the containers take their information from their retained state and recreate the file on the host system. Is the logic in precedence somewhere specified? – and0r Oct 22 '20 at 14:52
  • 1
    @and0r the folder is shared, imagine google drive and real users + shared folder, if someone delete file on that folder than it's gone for the rest of users – Andriy Ivaneyko Oct 22 '20 at 18:12
  • 5
    @AndriyIvaneyko Are named volumes actually removed? Couldn't find any references on that. – Kevin Van Ryckegem Mar 22 '21 at 14:32
  • Named values are not removed, at least as of May 2023, check https://docs.docker.com/compose/compose-file/07-volumes/ – ᐅdevrimbaris May 05 '23 at 11:30
  • kevinVanRyckegem @devrimbaris good point, I've updated answer, thank you folks. – Andriy Ivaneyko May 05 '23 at 19:55
12

Previous answers helped me a lot (thanks!) but it took some more time to figure out how to set options to create a shared tmpfs volume (memory file system). I'm sharing it in the hope to make life easier for developers in the same situation.

An example showing only the relevant parts of docker-compose.yml:

version: '3'

volumes:
  shared-tmpfs:
    driver: local
    driver_opts:
      type: "tmpfs"
      device: "tmpfs"
      o: "size=256m,uid=1000"

services:
  nginx:
    volumes:
      - shared-tmpfs:/tmp/mytmpfs

  php-fpm:
    volumes:
      - shared-tmpfs:/tmp/mytmpfs

I use it to spare my SSD from heavy writes (building lot's of static html files) in development/watch mode.

You can read more about driver_opts in the official Docker docs here.

szegheo
  • 4,175
  • 4
  • 31
  • 35
  • What about the accepted answer didn't work for you? Here you have to define it twice - with the accepted answer's `&` and `*` notation so you only have to define it once? – Jimbo Mar 23 '21 at 09:32
  • 1
    @Jimbo I do like the accepted answer but I wasn't able to achieve the same with that approach. That solution binds an existing directory, my solution creates a new `tmpfs` volume on-the-fly with the given options. It may be my fault, but I wasn't able to convert it to the form of the accepted answer (extension fields). – szegheo Mar 23 '21 at 13:55