How to kubectl wait for crd creation?

Question

What is the best method for checking to see if a custom resource definition exists before running a script, using only kubectl command line?

We have a yaml file that contains definitions for a NATS cluster ServiceAccount, Role, ClusterRoleBinding and Deployment. The image used in the Deployment creates the crd, and the second script uses that crd to deploy a set of pods. At the moment our CI pipeline needs to run the second script a few times, only completing successfully once the crd has been fully created. I've tried to use kubectl wait but cannot figure out what condition to use that applies to the completion of a crd.

Below is my most recent, albeit completely wrong, attempt, however this illustrates the general sequence we'd like.

kubectl wait --for=condition=complete kubectl apply -f 1.nats-cluster-operator.yaml kubectl apply -f 2.nats-cluster.yaml

score 17 · Accepted Answer · answered Jul 20 '19 at 16:15

17

The condition for a CRD would be established:

kubectl -n <namespace-here> wait --for condition=established --timeout=60s crd/<crd-name-here>

You may want to adjust --timeout appropriately.

answered Jul 20 '19 at 16:15

apisim

4,036
1
10
16

8

I find that I still get `Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "environments.fission.io" not found ` with this command occasionally, so I think waiting for just the `established` condition is insufficient. Anyone aware of a better way of doing this that is resilient to races with the CRD's creation? – hornairs Jan 12 '21 at 03:03
@hornairs -- have you been able to determine why the CRD wasn't created? if it fails only (quote) "...occasionally" - is it possible that the `--timeout` argument needs to be increased? – apisim Jan 27 '21 at 01:21
2

This fails for me `kubectl wait --for condition=established --timeout=60s crd/servicemonitors.monitoring.coreos.com` with `Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "servicemonitors.monitoring.coreos.com" not found` – geekQ Apr 15 '21 at 18:06
`while ! kubectl wait --for condition=established --timeout=60s crd/ ; do` `sleep 1` `done` should cover also `error: unable to recognize ... no matches for kind ... in version ...` error – mj41 Sep 16 '22 at 12:08

score 2 · Answer 2 · answered May 19 '21 at 21:27

In case you are wanting to wait for a resource that may not exist yet, you can try something like this:

{ grep -q -m 1 "crontabs.stable.example.com"; kill $!; } < <(kubectl get crd -w)

or

{ sed -n /crontabs.stable.example.com/q; kill $!; } < <(kubectl get crd -w)

I understand the question would prefer to only use kubectl, however this answer helped in my case. The downside to this method is that the timeout will have to be set in a different way and that the condition itself is not actually checked.

In order to check the condition more thoroughly, I made the following:

#!/bin/bash

condition-established() {
    local name="crontabs.stable.example.com"
    local condition="Established"

    jq --arg NAME $name --arg CONDITION $condition -n \
        'first(inputs | if (.metadata.name==$NAME) and (.status.conditions[]?.type==$CONDITION) then
            null | halt_error else empty end)' 

    # This is similar to the first, but the full condition is sent to stdout
    #jq --arg NAME $name --arg CONDITION $condition -n \
    #    'first(inputs | if (.metadata.name==$NAME) and (.status.conditions[]?.type==$CONDITION) then
    #        .status.conditions[] | select(.type==$CONDITION) else empty end)' 
}

{ condition-established; kill $!; } < <(kubectl get crd -w -o json)

echo Complete

To explain what is happening, $! refers to the command run by bash's process substitution. I'm not sure how well this might work in other shells.

I tested with the CRD from the official kubernetes documentation.

How to kubectl wait for crd creation?

2 Answers2