1

Is it safe / possible to use flock in a docker swarm?

I'm looking for a safe way for multiple containers to write to the same file (on the same volume). I know that on a single host docker will bind mount. What I'm not certain about is how this works when spinning up containers via docker-compose on a swarm.

If I have multiple instances of the same image in a service and these all share a volume then when I start this on a swarm will the containers be started on separate hosts? If so how will the the volume be shared at an OS level? What synchronisation options do I have for controlling concurrent writes?

Philip Couling
  • 13,581
  • 5
  • 53
  • 85
  • 1
    You should output your data to an [NFS backed volume](https://stackoverflow.com/a/57742017/1423507) that is available to all the nodes of the swarm cluster and manages disk access. Bind-mounts are not synced between nodes. – masseyb Oct 07 '20 at 10:59

2 Answers2

2

Create a NFS volume

Example:

  # create a reusable volume
  $ docker volume create --driver local \
      --opt type=nfs \
      --opt o=nfsvers=4,addr=192.168.1.1,rw \
      --opt device=:/path/to/dir \
      foo

  # inside a docker-compose file
  ...
  volumes:
    nfs-data:
      driver: local
      driver_opts:
        type: nfs
        o: nfsvers=4,addr=192.168.1.1,rw
        device: ":/path/to/dir"
Bilal Ali Jafri
  • 915
  • 1
  • 6
  • 17
  • I'm a bit confused by this answer. The question was about *synchronisation* and Docker's behaviour. In that context this answer says nothing of synchronisation or Docker's (default?) behaviour. – Philip Couling Oct 07 '20 at 13:10
  • @PhilipCouling it's a start (lacks details) but essentially you're asking 2 questions: (1) how volumes can be shared between `swarm` nodes and (2) synchronization options for concurrent writes - NFS addresses sharing volumes between nodes and [implements close-to-open consistency](https://serverfault.com/a/73927/526382). – masseyb Oct 07 '20 at 13:26
  • 1
    @masseyb no I'm not asking how that *can* be shared. That is not in my question at all. My first question what a "what happens if"... will they actually be started on separate hosts? And to be clear I wasn't asking about bind mounts (confusingly enough). I was asking about simple volumes. – Philip Couling Oct 07 '20 at 13:43
  • @PhilipCouling ok, must have misunderstood "I start this on a swarm will the containers be started on separate hosts? If so how will the the volume be shared at an OS level?" - yes they will be started on separate hosts. Simple [volumes are bind mounts](https://docs.docker.com/storage/volumes/) and the volume is created on whichever host the container ends up on. `docker` volumes are NOT sync'd between nodes. Confusing indeed. If you need access to a same volume across multiple nodes then you can use NFS. – masseyb Oct 07 '20 at 13:47
  • @PhilipCouling "I'm looking for a safe way for multiple containers to write to the same file (on the same volume)" "in a docker swarm" - please clarify your question if you are not actually looking for a way to synchronize concurrent writes to the same volume in a `docker swarm`. Just the mention of `swarm` and "same volume" implies you'll need a shared volume between the `swarm` nodes and there's only so many ways you can do that and simple bind-mount `docker` volumes isn't one of them. – masseyb Oct 07 '20 at 14:06
  • 1
    @masseyb that's fine, it's becoming more a structural issue with the answer. Based off your comments, the answer needs three parts 1) explaining your comments above 2) explaining how to get round this limitation, *this is answered*. 3) discussing synchronisation options over NFS. – Philip Couling Oct 07 '20 at 14:31
  • @PhilipCouling let me try and summarize all that in an answer. – masseyb Oct 07 '20 at 14:32
1

docker volumes are local to the node on which they are created, they are not shared between docker swarm nodes.

When running in a multi-node swarm cluster (or in Kubernetes) the container can end up on any of the nodes inside the cluster. For shared access to a docker volume (or PVC in the case of Kubernetes) the volume must be backed by something like a Network File System that can be accessed from each of the nodes inside the cluster.

NFS version 4 implements close-to-open consistency. Full details of what NFS v4 implements are available in "9.3. Data Caching" of RFC 3530:

Share reservations and record locks are the facilities the NFS version 4 protocol provides to allow applications to coordinate access by providing mutual exclusion facilities.

Tip: named docker volumes are bind-mounts, the volumes are created on disk in (by default) /var/lib/docker/volumes and bind-mounted during the docker run -v <named_docker_volume>:<container_mount_path> ....

masseyb
  • 3,745
  • 1
  • 17
  • 29