How to scale a Kubernetes deployment which uses a Persistent Volume Claim to 2 Pods?

Question

I have a Kubernetes deployment (apache flume to be exact) which needs to store persistent data. It has a PVC set up and bind to a path, which works without problem.

When I simply increase the scale of the deployment through kubernetes dashboard, it gives me an error saying multiple pods are trying to attach the same persistent volume. My deployment description is something like this (I tried to remove irrelevant parts)

{
    "kind": "Deployment",
    "apiVersion": "extensions/v1beta1",
    "metadata": {
        "name": "myapp-deployment",    
        "labels": {
          "app": "myapp",
          "name": "myapp-master"
        },    
    "spec": {
    "replicas": 1,
    "selector": {
      "matchLabels": {
        "app": "myapp",
        "name": "myapp-master"
      }
    },
    "template": {
      "spec": {
        "volumes": [
          {
            "name": "myapp-data",
            "persistentVolumeClaim": {
              "claimName": "myapp-pvc"
            }
          }
        ],
        "containers": [
          {
            "name": "myapp",
            "resources": {},
            "volumeMounts": [
              {
                "name": "ingestor-data",
                "mountPath": "/data"
              }
            ]
          }
        ]
      }
    },...

Each pod should get its own persistent space (but with same pathname), so one doesn't mess with the others'. I tried to add a new volume in the volume array above, and a volume mount in the volume mount array, but it didn't work (I guess it meant "bind two volumes to a single container")

What should I change to have 2 pods with separate persistent volumes? What should I change to have N number of pods and N number of PVC's so I can freely scale the deployment up and down?

Note: I saw a similar question here which explains N number of pods cannot be done using deployments. Is it possible to do what I want with only 2 pods?

Chris · Answer 1 · 2021-01-20T06:53:51.497

You should use a StatefulSet for that. This is for pods with persistent data that should survive a pod restart. Replicas have a certain order and are named in that way (my-app-0, my-app-1, ...). They are stopped and restarted in this order and will mount the same volume after a restart/update.

With the StatefulSet you can use a volumeClaimTemplates to dynamically create new PersistentVolumes with the creation of a new pod. So every time a pod is created a volume get provisioned by your storage class.

From docs:

The volumeClaimTemplates will provide stable storage using PersistentVolumes provisioned by a PersistentVolume Provisioner

volumeClaimTemplates:
  - metadata:
      name: www
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "my-storage-class"
      resources:
        requests:
          storage: 1Gi

See docs for more details: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#components

score 2 · Answer 2 · answered Jan 19 '21 at 18:13

Each pod should get its own persistent space (but with same pathname), so one doesn't mess with the others'.

For this reason, use a StatefulSet instead. Most things will work the same way, except that each Pod will get its own unique Persistent Volume.

How to scale a Kubernetes deployment which uses a Persistent Volume Claim to 2 Pods?

2 Answers2