Multiple Volume mounts with Kubernetes: one works, one doesn't

Question

I am trying to create a Kubernetes pod with a single container which has two external volumes mounted on it. My .yml pod file is:

apiVersion: v1
kind: Pod
metadata:
  name: my-project
  labels:
    name: my-project
spec:
  containers:
    - image: my-username/my-project
      name: my-project
      ports:
        - containerPort: 80
          name: nginx-http
        - containerPort: 443
          name: nginx-ssl-https
      imagePullPolicy: Always
      volumeMounts:
        - mountPath: /home/projects/my-project/media/upload
          name: pd-data
        - mountPath: /home/projects/my-project/backups
          name: pd2-data
  imagePullSecrets:
    - name: vpregistrykey
  volumes:
    - name: pd-data
      persistentVolumeClaim:
        claimName: pd-claim
    - name: pd2-data
      persistentVolumeClaim:
        claimName: pd2-claim

I am using Persistent Volumes and Persisten Volume Claims, as such: PV

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pd-disk
  labels:
    name: pd-disk
spec:
  capacity:
    storage: 250Gi
  accessModes:
    - ReadWriteOnce
  gcePersistentDisk:
    pdName: "pd-disk"
    fsType: "ext4"

PVC

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pd-claim
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 250Gi

I have initially created my disks using the command: $ gcloud compute disks create --size 250GB pd-disk Same goes for the second disk and second PV and PVC. Everything seems to work ok when I create the pod, no errors are thrown. Now comes the weird part: one of the paths is being mounted correctly (and is therefor persistent) and the other one is being erased every time I restart the pod...

I have tried re-creating everything from scratch, but nothing changes. Also, from the pod description, both volumes seem to be correctly mounted:

$ kubectl describe pod my-project
Name:       my-project
...
Volumes:
  pd-data:
    Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  pd-claim
    ReadOnly: false
  pd2-data:
    Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  pd2-claim
    ReadOnly: false

Any help is appreciated. Thanks.

What version of kubernetes do you use and how did you restart the pod? — Yu-Ju Hong, Feb 16 '16 at 23:20
I am using Kubernetes v1, and for restarting the pod I am using `kubectl delete -f my-project.yml` followed by `kubectl create -f my-project.yml` — AdelaN, Feb 16 '16 at 23:25
`Client Version: version.Info{Major:"1", Minor:"1", GitVersion:"v1.1.4", GitCommit:"a5949fea3a91d6a50f40a5684e05879080a4c61d", GitTreeState:"clean"} Server Version: version.Info{Major:"1", Minor:"1", GitVersion:"v1.1.7", GitCommit:"e4e6878293a339e4087dae684647c9e53f1cf9f0", GitTreeState:"clean"}` — AdelaN, Feb 17 '16 at 09:16
If you were able to solve this, you can post self-answer here for other community members who may be seeing this same issue. — Faizan, Feb 27 '17 at 21:29
Sorry to let you down, but I was unable to find a solution last year, so I just found an alternative which involved a single volume being mounted... — AdelaN, Feb 28 '17 at 22:24
Are you sure that your PVCs got actually bound to the correct PVs? Because since you are doing manual provisioning, you should first specify the `claimRef` on the PV object, which points to your PVC, and then create the PVC with that name. I do not see `claimRef` in your examples, neither the full resources for reproducing it (seconds claim and disk). If you apply the PVC example you have as-is, it will get dynamically provisioned by the default storage class, which is not the bahavior you expect (since you create your disk manually via `gcloud` command) — Utku Özdemir, May 22 '19 at 22:23

score 5 · Answer 1 · answered Mar 19 '19 at 10:32

The Kubernetes documentation states:

Volumes can not mount onto other volumes or have hard links to other volumes

I had the same issue and in my case the problem was that both volume mounts had overlapping mountPaths, i.e. both started with /var/.

They mounted without issues after fixing that.

score 4 · Answer 2 · edited Mar 03 '20 at 15:24

I do not see any direct problem for which such behavior as explained above has occurred! But what I can rather ask you to try is to use a "Deployment" instead of a "Pod" as suggested by many here, especially when using PVs and PVCs. Deployment takes care of many things to maintain the "Desired State". I have attached my code below for your reference which works and both the volumes are persistent even after deleting/terminating/restarting as this is managed by the Deployment's desired state.

Two difference which you would find in my code from yours are:

I have a deployment object instead of pod
I am using GlusterFs for my volume.

Deployment yml.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx
  namespace: platform
  labels:
    component: nginx
spec:
  replicas: 2
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
  template:
    metadata:
      labels:
        component: nginx
    spec:
      nodeSelector:
        role: app-1
      containers:
        - name: nginx
          image: vip-intOAM:5001/nginx:1.15.3
          imagePullPolicy: IfNotPresent
          volumeMounts:
          - mountPath: "/etc/nginx/conf.d/"
            name: nginx-confd
          - mountPath: "/var/www/"
            name: nginx-web-content
      volumes:
      - name: nginx-confd
        persistentVolumeClaim:
          claimName: glusterfsvol-nginx-confd-pvc
      - name: nginx-web-content
        persistentVolumeClaim:
          claimName: glusterfsvol-nginx-web-content-pvc

One of my PV

apiVersion: v1
kind: PersistentVolume
metadata:
  name: glusterfsvol-nginx-confd-pv
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  glusterfs:
    endpoints: gluster-cluster
    path: nginx-confd
    readOnly: false
  persistentVolumeReclaimPolicy: Retain
  claimRef:
    name: glusterfsvol-nginx-confd-pvc
    namespace: platform

PVC for the above

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: glusterfsvol-nginx-confd-pvc
  namespace: platform
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

score 0 · Answer 3 · answered Jun 07 '23 at 07:34

0

One possible reason is that the volumeHandle in the two PVs are same, which will make it fail to mount both PVs into same pod.

answered Jun 07 '23 at 07:34

Hongbo Liu

2,818
1
24
18

Multiple Volume mounts with Kubernetes: one works, one doesn't

3 Answers3

Linked