How to forbid Pods in Namespaces without ResourceQuota using Gatekeeper?

Question

I want to forbid the creation of Pods in Namespaces which do not have a ResourceQuota. If possible, I want Gatekeeper to ensure that there is a ResourceQuota which sets limits.cpu and limits.memory before allowing the creation of Pods.

I have created below configurations, but they have not solved my problem:

template

apiVersion: templates.gatekeeper.sh/v1beta1 
kind: ConstraintTemplate 
metadata: 
  name: k8sresoucequota 
spec:
  crd:
    spec:
      names:
        kind: k8sResouceQuota
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8sresoucequota

        violation[{"msg": msg}] {
          input.request.kind.kind == "Pod"
          requestns := input.request.object.metadata.namespace
          existingrqs := {e | e := data.inventory.namespace[requestns]["v1beta1"]["ResourceQuota"].metadata.name}
          not ns_exists(requestns,existingrqs)
          msg := sprintf("container <%v> could not be created because the <%v> namespace does not have ResourceQuotas defined", [input.request.object.metadata.name,input.request.object.metadata.namespace])
        }

        ns_exists(ns,arr) {
          arr[_] = ns
        }

Constraint

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: k8sResouceQuota
metadata:
  name: namespace-must-have-resourcequota
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
    excludedNamespaces:
      - kube-system
      - kube-public
      - kube-node-lease
      - default
      - gatekeeper-system
      - kubernetes-dashboard

sync.yaml

apiVersion: config.gatekeeper.sh/v1alpha1
kind: Config
metadata:
  name: config
  namespace: "gatekeeper-system"
spec:
  sync:
    syncOnly:
      - group: ""
        version: "v1beta1"
        kind: "Pod"
      - group: ""
        version: "v1beta1"
        kind: "Namespace"
      - group: ""
        version: "v1beta1"
        kind: "ResourceQuota"

score 0 · Answer 1 · answered May 15 '21 at 02:44

Part 1: "There is no ResourceQuota with the same name as the Namespace"

Let's walk through these three lines of your Rego, one-by-one.

requestns := input.request.object.metadata.namespace
existingrqs := {e | e := data.inventory.namespace[requestns]["v1beta1"]["ResourceQuota"].metadata.name}
not ns_exists(requestns,existingrqs)

This line gets the name of the Namespace which holds the Pod being evaluated.

requestns := input.request.object.metadata.namespace

This line is invalid. Or rather, it will return an empty set.

existingrqs := {e | e := data.inventory.namespace[requestns]["v1beta1"]["ResourceQuota"].metadata.name}

I assume what you're trying to say is "get the name of every ResourceQuota in the same Namespace as the Pod.

Recall that the format of data is data.inventory.namespace[<namespace>][groupVersion][<kind>][<name>]. So really the line should be:

existingrqs := {e | e := data.inventory.namespace[requestns]["v1beta1"]["ResourceQuota"][_].metadata.name

This will iterate through all ResourceQuotas in the Pod's Namespace, and create a set containing all of their names.

You have this split out as it's own function (ns_exists), but it looks like you're trying to find whether there is a ResourceQuota with the same name as the Namespace. Instead, you should use:

not existingrqs[requestns]

This statement will evaluate to false if the ResourceQuota of the same name exists, and true if it does not. But this is actually doing more than we need. We just care whether the ResourceQuota exists. We can combine lines 2 and 3 into:

not data.inventory.namespace[requestns]["v1beta1"]["ResourceQuota"][requestns]

This will evaluate to true if the ResourceQuota is missing, or false if there is such a ResourceQuota

The completed Rego should be:

package k8sresoucequota

violation[{"msg": msg}] {
  input.request.kind.kind == "Pod"
  requestns := input.request.object.metadata.namespace
  not data.inventory.namespace[requestns]["v1beta1"]["ResourceQuota"][requestns]
  msg := sprintf("container <%v> could not be created because the <%v> namespace does not have ResourceQuotas defined", [input.request.object.metadata.name,input.request.object.metadata.namespace])
}

In English, this will create a violation if the object is a Pod, and the Namespace does not contain a ResourceQuota of the same name as the Namespace.

Part 2: "The ResourceQuota sets limits.cpu and limits.memory"

For Rego, we use incremental rules to create OR conditions. While possible, it's considered an antipattern to circumvent that all statements in a Rego function are AND-ed together. So we need a different violation that checks that the ResourceQuota has limits.cpu and limits.memory set.

violation[{"msg": msg}] {
  input.request.kind.kind == "Pod"
  requestns := input.request.object.metadata.namespace
  data.inventory.namespace[requestns]["v1beta1"]["ResourceQuota"][requestns]
  rq := data.inventory.namespace[requestns]["v1beta1"]["ResourceQuota"][requestns]
  not defines_limits(rq)
  msg := sprintf("container <%v> could not be created because the <%v> namespace does not enforce limits.cpu or limits.memory", [input.request.object.metadata.name,input.request.object.metadata.namespace])
}

defines_limits(rq) {
  rq.limits.cpu
}

defines_limits(rq) {
  rq.limits.memory
}

This violation checks:

That a ResourceQuota of the desired name exists.
The ResourceQuota fails to set one of limits.cpu and limits.memory. That is, the ResourceQuota we found must set both in order to allow the Pod to be created.

This answer does a good job explaining how incremental rules work.

How to forbid Pods in Namespaces without ResourceQuota using Gatekeeper?

1 Answers1