72

I am trying to delete persistent volumes on a Kubernetes cluster. I ran the following command:

kubectl delete pv pvc-08e65270-b7ce-11e9-ba0b-0a1e280502e2 pvc-08e87826-b7ce-11e9-ba0b-0a1e280502e2 pvc-08ea5f97-b7ce-11e9-ba0b-0a1e280502e2 pvc-08ec1cac-b7ce-11e9-ba0b-0a1e280502e2

However it showed:

persistentvolume "pvc-08e65270-b7ce-11e9-ba0b-0a1e280502e2" deleted
persistentvolume "pvc-08e87826-b7ce-11e9-ba0b-0a1e280502e2" deleted
persistentvolume "pvc-08ea5f97-b7ce-11e9-ba0b-0a1e280502e2" deleted
persistentvolume "pvc-08ec1cac-b7ce-11e9-ba0b-0a1e280502e2" deleted

But the command did not exit. So I CONTROL+C to force exit the command. After a few minutes, I ran:

kubectl get pv

And the status is Terminating, but the volumes don't appear to be deleting.

How can I delete these persistent volumes?

Justin
  • 42,716
  • 77
  • 201
  • 296
  • Which provisioner do you use? I think the command is just waiting your provisioner to delete the volume, it looks like something goes wrong. Please provide the log from kubelet, or log from provisioner if you use kubernetes-csi. – menya Aug 08 '19 at 03:31
  • 1
    In my case the trick was not only to delete the 'claims' but also the relate 'persistent volumes' (the pods were already gone since I got rid of the related 'services', 'deployments' etc.). Thanks for asking! – muthuh Jan 17 '22 at 09:38

4 Answers4

67

It is not recommended to delete pv it should be handled by cloud provisioner. If you need to remove pv just delete pod bounded to claim and then pvc. After that cloud provisioner should also remove pv as well.

kubectl delete pvc --all 

It sometimes could take some time so be patient.

DArkO
  • 15,880
  • 12
  • 60
  • 88
FL3SH
  • 2,996
  • 1
  • 17
  • 25
  • The pods are deleted. They were originally created by a Helm chart. I don't think when I did `helm delete` it deletes the PV volumes. The PV volumes are still showing `Terminating`, been over 40 minutes. – Justin Aug 07 '19 at 20:42
  • Once I deleted pv on gke and it took more than an hour for kubernetes to figure out what happened. – FL3SH Aug 07 '19 at 21:19
  • @Justin `helm delete` only deletes the pvc and it depends on the reclaim policy of the pv to determine whether it is retained or deleted – Hang Du Aug 08 '19 at 01:14
  • 1
    @FL3SH Please fix the type in `kubectl` – Mohamed Saleh Jan 04 '20 at 17:37
  • had no issues with this on v1.21 – Glitch May 19 '22 at 16:22
39

Delete all the pods, which is using the pvc(you want to delete), then delete the PVC(PersistentVolumeClaim) & PV(PersistentVolume) in sequence.

Some thing like below(in sequence):

  1. kubectl delete pod --all / pod-name
  2. kubectl delete pvc --all / pvc-name
  3. kubectl delete pv --all / pv-name
OTStats
  • 1,820
  • 1
  • 13
  • 22
Shreyas
  • 549
  • 5
  • 3
  • I deleted my PV by mistake, when I tried to get all pv in specific namespace, it's returned all pv in my cluster. So I thought it was pv in this namespace, so I deleted all of these PV. But luckily, these PV won't get delete by the above rules. Feeling relieve ^^ – Ethan Nguyen Jun 19 '21 at 03:16
29

I have created below diagram to help explain this better.

enter image description here

The Kubectl commands are mentioned by other answers in this thread. The same should work.

kubectl delete sts sts-name
kubectl delete pvc pvc-name
kubectl delete pv pv-name 

Some more useful info

  • If you see something stuck in terminating state, its because of guardrails set in place by k8s. These are referred to as 'Finalizers'.

  • If your PV is stuck in terminating state after deletion, it likely because you have deleted the PV before deleting the PVC.

  • If your PVC is stuck in terminating state after deletion, it likely because your pods are still running. (simply delete the pods/statefulset in such cases)

  • If you wish to delete the resource in terminating state, use below commands to bypass the pvc, pv protection finalizers.

kubectl patch pvc pvc_name -p '{"metadata":{"finalizers":null}}'

kubectl patch pv pv_name -p '{"metadata":{"finalizers":null}}'


ns15
  • 5,604
  • 47
  • 51
7

PVs are cluster resources provisioned by an administrator, whereas PVCs are a user's request for storage and resources. I guess you have still deployed the corresponding PVC.

  1. Delete the deployment. E.g.:

    enter image description here

    kubectl delete deployment mongo-db
    
  2. List the Persistent Volume Claim. E.g.:

    enter image description here

    kubectl get pvc
    
  3. Delete the corresponding pcv. E.g.:

    enter image description here

    kubectl delete pvc mongo-db
    
sfl0r3nz05
  • 547
  • 8
  • 14