Checkov - creating a basic custom policy to ensure that an annotation is set on a Kubernetes Deployment

Question

I've been looking at checkov to see if it can flag up if any Kubernetes Deployments which are missing the annotation kubectl.kubernetes.io/default-container.

I cannot seem to get this to work. It seems like a very simple use case for checkov.

I've currently got the following policy document:

---
metadata:
  id: "CKV2_KCDC_1"
  name: "Ensure all Deployments have default-container annotation"
  category: "KUBERNETES"
definition:
  and:
    - cond_type: filter
      value:
        - Deployment
      operator: within
      attribute: kind
    - cond_type: attribute
      resource_types:
        - Deployment
      attribute: "metadata.annotations.kubectl.kubernetes.io/default-container"
      operator: exists

My interpretation of this is "Filter for Deployments, and ensure that each one has the annotation"

When I run this, I get a lot of failures, but when I add the annotation to the failing manifests those failures are not resolved.

I'm not famliar with Checkov, but `metadata.annotations.kubectl.kubernetes.io/default-container` looks problematic; you're using `.` as both the hierarchy separator **and** as part of the field value (and there's no way to tell which is which). Maybe the docs address how to escape things properly? — larsks, Jun 08 '23 at 12:39
I did think that too, I haven't seen anything on escaping yet. I also realise that I should be checking for this annotation inside the template section of the manifest rather than on the deployment itself, but I can revisit that once I have something that works. — Scottm, Jun 08 '23 at 13:07
I've worked in the past with [gatekeeper](https://github.com/open-policy-agent/gatekeeper), which supports syntax like `'metadata.annotations."cnrm.cloud.google.com/deletion-policy"'`. — larsks, Jun 08 '23 at 13:51

score 0 · Accepted Answer · answered Jun 14 '23 at 13:13

I ended up going with datree for this. My organisation was already using it, and I found it very easy to write a policy with a custom rule for my scenario. The policy looks something like this:

apiVersion: v1
policies:
  - name: Custom
    isDefault: true
    rules:
      - identifier: ENSURE_DEFAULT_CONTAINER_ANNOTATION_IS_SET
        messageOnFailure: Every workload must set the kubectl.kubernetes.io/default-container annotation so that multi-container workloads have sensible defaults for kubctl exec and kubectl log commands.
customRules:
  - identifier: ENSURE_DEFAULT_CONTAINER_ANNOTATION_IS_SET
    name: Ensure workload has default container annotation set
    defaultMessageOnFailure: Every workload must set the kubectl.kubernetes.io/default-container annotation so that multi-container workloads have sensible defaults for kubctl exec and kubectl log commands.
    schema:
      if:
        properties:
          kind:
            enum:
              - Deployment
              - StatefulSet
      then:
        properties:
          spec:
            properties:
              template:
                properties:
                  metadata:
                    properties:
                      annotations:
                        required:
                          - kubectl.kubernetes.io/default-container
                    required:
                      - annotations

Checkov - creating a basic custom policy to ensure that an annotation is set on a Kubernetes Deployment

1 Answers1