36

It seems that Kubernetes supports 3 kinds of access mode for persistent volume: ReadWriteOnce, ReadOnlyMany, ReadWriteMany. I'm really curious about the scheduler strategy for a pod which uses the ReadWriteOnce mode volume. For example, I created an RC which have pod num=2, I guess the two pods will be scheduled into the same host because they use the volume that has ReadWriteOnce mode? I really want to know the source code of this part.

Jonas
  • 121,568
  • 97
  • 310
  • 388
wangzhe
  • 573
  • 1
  • 5
  • 13

3 Answers3

27

I think DavidO's answer is wrong. As per Kubernetes docs on Access Modes:

The access modes are:

  • ReadWriteOnce -- the volume can be mounted as read-write by a single node
  • ReadOnlyMany -- the volume can be mounted read-only by many nodes
  • ReadWriteMany -- the volume can be mounted as read-write by many nodes

So AccessModes as defined today, only describe node attach (not pod mount) semantics, and doesn't enforce anything.

So to prevent two pods mount the same PVC even if they are scheduled to be run on the same node you can use pod anti-affinity. It is not the same as not to mount one volume to 2 pods scheduled on the same node. But anti-affinity can be used to ask scheduler not to run 2 pods on the same node. Therefore it prevents mounting one volume into 2 pods.

Cody Gray - on strike
  • 239,200
  • 50
  • 490
  • 574
0xMH
  • 1,825
  • 20
  • 26
11

If a pod mounts a volume with ReadWriteOnce access mode, no other pod can mount it. In GCE (Google Compute Engine) the only allowed modes are ReadWriteOnce and ReadOnlyMany. So either one pod mounts the volume ReadWrite, or one or more pods mount the volume ReadOnlyMany.

The scheduler will not allow a pod to schedule if it uses a GCE volume that has already been mounted read-write.

(Documentation reference: persistent volume access modes)

Cody Gray - on strike
  • 239,200
  • 50
  • 490
  • 574
DavidO
  • 1,647
  • 11
  • 7
  • Thanks for your answer! it seems that in `isVolumeConflict` func, only the mode of GCEPD is checked , what if I use the readwriteonce mode for other type of persistent storage like aws or nfs ? is there any algorithm for testing such conflicts in scheduler? – wangzhe Jun 06 '16 at 05:56
  • It seems that isVolumeConflic fun only support GCEPD,AWSElasticBlockStore,RBD checking currently? – wangzhe Jun 06 '16 at 07:21
  • 12
    "the volume can be mounted as read-write by a single node" I think ReadWriteOnce means this volume can be mounted on only one k8s node. I have tried mounted the same ReadWriteOnce volume on the same node by two pods. And it works – Haoyuan Ge Oct 18 '17 at 06:30
  • 2
    ive seen the same volume mounted on 2 pods, as well - only if on the same node though - in GKE dynamic provisioning – jayunit100 Apr 06 '18 at 06:24
  • 7
    As per the official documentation of Kubernetes volumes, the access mode ReadWriteOnce restricts the volume to be bounded by more than one NODE (not POD). – ACloudRoamer Sep 05 '19 at 03:51
  • 4
    Also, citing [Kubernetes in Action](https://www.manning.com/books/kubernetes-in-action): > `RWO`, `ROX`, and `RWX` pertain to the number of worker nodes that can use the volume at the same time, not to the number of pods! – LaloLoop Apr 20 '20 at 17:04
  • Note also that the restriction of storage access to just one POD is called `ReadWriteOnce**Pod**` and was introduced only in May 2023 (k8s v1.27), so 7 years after this answer (and it is distinct from "plain" `ReadWriteOnce` which refers to NODES). – mirekphd May 28 '23 at 10:05
  • Even [the docs](https://web.archive.org/web/20161116042557/http://kubernetes.io/docs/user-guide/persistent-volumes/) linked retrospectively (kudos to Cody Gray) from the Internet Archive do not support the thesis from this answer that "no other pod can mount it". In sect. `Persistent Volumes > Access Modes`" you can find that **"ReadWriteOnce – the volume can be mounted as read-write by a single node"**. Node, not a pod... And then we are blaming LLMs calling it "hallucinations"... – mirekphd May 28 '23 at 15:19
6

In Kubernetes you provision storage either statically(using a storage class) or dynamically (Persistent Volume). Once the storage is available to bound and claimed, you need to configure it in what way your Pods or Nodes are connecting to the storage (a persistent volume). That could be configured in below four modes.

ReadOnlyMany (ROX)
In this mode multiple pods running on different Nodes could connect to the storage and carry out read operation.

ReadWriteMany (RWX)
In this mode multiple pods running on different Nodes could connect to the storage and carry out read and write operation.

ReadWriteOnce (RWO)
In this mode multiple pods running in only one Node could connect to the storage and carry out read and write operation.

ReadWriteOncePod (RWOP)
In this mode the volume can be mounted as read-write by a single Pod. Use ReadWriteOncePod access mode if you want to ensure that only one pod across whole cluster can read that PVC or write to it. This is only supported for CSI volumes and Kubernetes version 1.22+.

Follow the documentation to get more insight.

Aditya Bhuyan
  • 328
  • 6
  • 10