8

I am getting issue while terminating the namesapce in the cluster, It's showing many parameters inside the namespace JSON. I followed this link https://medium.com/@craignewtondev/how-to-fix-kubernetes-namespace-deleting-stuck-in-terminating-state-5ed75792647e

 "spec": {},
    "status": {
        "conditions": [
            {
                "lastTransitionTime": "2021-01-11T08:41:48Z",
                "message": "All resources successfully discovered",
                "reason": "ResourcesDiscovered",
                "status": "False",
                "type": "NamespaceDeletionDiscoveryFailure"
            },
            {
                "lastTransitionTime": "2021-01-11T08:41:48Z",
                "message": "All legacy kube types successfully parsed",
                "reason": "ParsedGroupVersions",
                "status": "False",
                "type": "NamespaceDeletionGroupVersionParsingFailure"
            },
            {
                "lastTransitionTime": "2021-01-11T08:41:48Z",
                "message": "All content successfully deleted, may be waiting on finalization",
                "reason": "ContentDeleted",
                "status": "False",
                "type": "NamespaceDeletionContentFailure"
            },
            {
                "lastTransitionTime": "2021-01-11T08:42:09Z",
                "message": "All content successfully removed",
                "reason": "ContentRemoved",
                "status": "False",
                "type": "NamespaceContentRemaining"
            },
            {
                "lastTransitionTime": "2021-01-11T08:41:48Z",
                "message": "All content-preserving finalizers finished",
                "reason": "ContentHasNoFinalizers",
                "status": "False",
                "type": "NamespaceFinalizersRemaining"
            }
        ],
        "phase": "Terminating"
    }
}```
rajendra sharma
  • 449
  • 1
  • 5
  • 7
  • I've already seen this when there is a webhook still active. In this cases it took around 30 mins to delete the namespace. How long did you wait? – alexzimmer96 Jan 11 '21 at 13:39
  • Does this answer your question? [Namespace "stuck" as Terminating, How do I remove it?](https://stackoverflow.com/questions/52369247/namespace-stuck-as-terminating-how-do-i-remove-it) – Saikat Chakrabortty Jan 11 '21 at 14:06
  • Still in showing terminating since 215 min almost – rajendra sharma Jan 11 '21 at 14:57
  • @SaikatChakrabortty no it's not helping even I tried ```kubectl delete ns fleet-system --grace-period=0 --force --namespace -n fleet-system``` ```warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.``` warning: deleting cluster-scoped resources, not scoped to the provided namespace namespace "fleet-system" force deleted And stuck here------ – rajendra sharma Jan 12 '21 at 15:50

3 Answers3

27

I have found the answer to terminate the stuck namespace.

for ns in $(kubectl get ns --field-selector status.phase=Terminating -o jsonpath='{.items[*].metadata.name}')
do
  kubectl get ns $ns -ojson | jq '.spec.finalizers = []' | kubectl replace --raw "/api/v1/namespaces/$ns/finalize" -f -
done

for ns in $(kubectl get ns --field-selector status.phase=Terminating -o jsonpath='{.items[*].metadata.name}')
do
  kubectl get ns $ns -ojson | jq '.metadata.finalizers = []' | kubectl replace --raw "/api/v1/namespaces/$ns/finalize" -f -
done
Conrado
  • 1,402
  • 17
  • 23
rajendra sharma
  • 449
  • 1
  • 5
  • 7
10

Firstly export your namespace name in env which got struck in Terminating state

export NAMESPACE="monitoring"

Then run below command to delete the Terminating namespace

kubectl get namespace $NAMESPACE -o json   | tr -d "\n" | sed "s/\"finalizers\": \[[^]]\+\]/\"finalizers\": []/"   | kubectl replace --raw /api/v1/namespaces/$NAMESPACE/finalize -f -
Pratik Raj
  • 141
  • 1
  • 3
5

The tutorial you've used is not proper because deleting the namespace by removing finalizers is not good way to go since it could leave resources registered to a non existing namespace. Please take a look at this post: finalizer-kubernetes-ns.

You can try to find out which resources in the namespace are pending deletion by:

  • Finding all resources that still exist using command kubectl api-resources --verbs=list --namespaced -o name | xargs -n 1 kubectl get -n $yotur-ns-to-delete
  • Checking if any apiservice is unavailable and hence doesn't serve its resources by executing command kubectl get apiservice|grep False

Take a look also at this problem: ns-kubernetes-stuck-terminating.

Malgorzata
  • 6,409
  • 1
  • 10
  • 27
  • 1
    ```kubectl api-resources --verbs=list --namespaced -o name | xargs -n 1 kubectl get -n prometheus``` error: unable to retrieve the complete list of server APIs: custom.metrics.k8s.io/v1beta1: the server is currently unable to handle the request No resources found in prometheus namespace. ```kubectl get apiservice|grep False``` v1beta1.custom.metrics.k8s.io prometheus/prometheus-adapter False (ServiceNotFound) 60d – rajendra sharma Jan 12 '21 at 06:06
  • Try to execute command: $ kubectl delete apiservice v1beta1.custom.metrics.k8s.io – Malgorzata Jan 12 '21 at 10:52
  • Error from server (NotFound): apiservices.apiregistration.k8s.io "v1beta1.custom.metrics.k8s.io" not found – rajendra sharma Jan 12 '21 at 14:36
  • Can you also try $ kubectl api-resources --verbs=list --namespaced -o name \ | xargs -n 1 kubectl get --show-kind --ignore-not-found -l – Malgorzata Jan 12 '21 at 16:44
  • Thanks @Malgorzata I had wanted to clean up a prometheus-adapter PoC and blew away everything with a k delete -f but forgot my NS was in there. ``` kubectl get apiservice|grep False v1beta1.custom.metrics.k8s.io prometheus/g-prometheus-adaptor-prometheus-adapter False (ServiceNotFound) 5h9m ``` Deleting custom metrics api allowed my NS to finally terminate. ``` kubectl delete apiservice v1beta1.custom.metrics.k8s.io ``` – GorginZ Jul 03 '23 at 05:32