2

Say I have some special nodes in my cluster, and I want to be able to identify with a pod label all pods running on these nodes.

Taints & tolerations are close to this, but afaik tolerations aren’t labels, they’re their own thing, and can’t be referenced in places where you expect a label selector, right? Is there any way to require all pods running on these nodes to identify themselves in a way that can then be queried via label selectors?

alecbz
  • 6,292
  • 4
  • 30
  • 50
  • Yes taints and tolerations are their own thing completely different from labels. They have their own function during a scheduling cycle. To answer your question: Apply a taint to the node. Apply a label to the node. Add this taint's toleration to pods. Add the nodeSelector to the pod. Query pods by node rather than a label. Would this work? – ashu May 12 '21 at 18:57
  • Sort of, except nothing requires that the label need to be on the pod if it's running on that node. We can do this by convention, but I was hoping for a way to force it, so that nothing could run on these nodes without getting this label. But it seems like I might need to bite the bullet and use a custom webhook admission controller to ensure the label is there? – alecbz May 12 '21 at 20:35
  • You may be interested in [this](https://gmaslowski.com/kubernetes-node-label-to-pod/) as a workaround. – ashu May 13 '21 at 03:27

1 Answers1

1

There are two parts of problem to solve:
1. Ensure that pods with specific labels are scheduled to specific set of nodes.
This can be done by using custom scheduler or a plugin for default scheduler (more here, here, here, here and here).

2. Prevent other schedulers to schedule pods to that nodes
This can by achieved by using nodeAffinity or node taints (more here and here)

A complete solution may require additional apiserver admission controller to set all those properties to a pod that suppose to have a specific label only.
Unfortunately, there is no easy built-in solution as of yet. Hopefuly the above will point you in the right direction.