Watch event for specific object

Question

We have an k8s operator (based on kubebuilder) which works as expected, now we need support for listening to secrets on the cluster.

The following code is working however I got event for all the secrets in the cluster which is not efficient,

I WANT to get the event only for specific secret, lets say secret with specific labels/annotation, how we can do it?

func (r *InvReconciler) SetupWithManager(mgr ctrl.Manager) error {
    manager := ctrl.NewControllerManagedBy(mgr).
        For(&corev1alpha1.Inv{}, builder.WithPredicates(predicate.Or(predicate.GenerationChangedPredicate{}, predicate.AnnotationChangedPredicate{}))).
        WithOptions(controller.Options{
        })

    manager = manager.Watches(&source.Kind{Type: &v1.Secret{}}, handler.EnqueueRequestsFromMapFunc(func(a client.Object) []reconcile.Request {
        return r.secretHandler.GetSecret(a.GetNamespace(), a.GetName())
    }))

    return manager.Complete(r)
}

this is the function

func (secretReq secretHandler) GetSecret(namespace string, name string) []reconcile.Request {

    fmt.Println("secret is: ", namespace, "--", name)
    return nil
}

lets say secret like the following and only for this secret (with labels foo: bar )i'll get the event when it was created or modified

apiVersion: v1
kind: Secret
metadata:
  labels:
    foo: bar
  name: mysecret
  namespace: dev
type: Opaque
data:
  USER_NAME: YWRtaW4=
  PASSWORD: dGVzdBo=

Im not talking about an if statement after I got the event as it already bring all the secrets event in the cluster.

Answer 1

According to this github source, you should be able to select specific objects (e.g. secrets) with EnqueueRequestForObject. However, it is not possible (yet) to watch only for specific secret CRUD changes.

EnqueueRequestForObject to watch for your CRD resource changes. In your CRD reconciler, you'd fetch all of the TLS secrets using a label selector based on the search definition and then run your merge logic with the matched secrets.

EnqueueRequestFromMapFunc to watch for secret changes and trigger a reconcile of one or more CRs. In your mapper function, you'd fetch all of the CRs. For each CR that has a search definition that matches the passed in secret, you'd create a new reconcile.Request for the CR, and return the list of requests, which would trigger your CRD reconciler for each CR that matched.

The cleanest way is using a label selector and then merge the results with your existing code. An example of using a label selector is given in this post:

func GetSecret(version string) (retVal interface{}, err error){
  clientset := GetClientOutOfCluster()
  labelSelector := metav1.LabelSelector{MatchLabels: map[string]string{"version":version}}

  listOptions := metav1.ListOptions{
    LabelSelector: labels.Set(labelSelector.MatchLabels).String(),
    Limit:         100,
  }
  secretList, err := clientset.CoreV1().Secrets("namespace").List(listOptions)
  retVal = secretList.Items[0]
  return retVal, err
}

Answer 2

This is possible, in a roundabout way. You need to configure the selector into the controller Manager that you use to set up your reconciler.

You can use label or field selectors for this. You can either set the same selector for all types of objects using DefaultSelector, or you can use SelectorsByObject to have different selectors for different types of objects.

import (
    corev1 "k8s.io/api/core/v1"
    "k8s.io/apimachinery/pkg/labels"
    "k8s.io/apimachinery/pkg/selection"
    "k8s.io/client-go/rest"
    ctrl "sigs.k8s.io/controller-runtime"
    "sigs.k8s.io/controller-runtime/pkg/cache"
    "sigs.k8s.io/controller-runtime/pkg/manager"
)

func startReconciler(cnf *rest.Config) error {
    mgr, err := ctrl.NewManager(cnf, manager.Options{
        NewCache: func(conf *rest.Config, opts cache.Options) (cache.Cache, error) {
            // Use this selector for everything that is not mentioned in SelectorsByObject
            opts.DefaultSelector = cache.ObjectSelector{
                Label: labels.SelectorFromSet(labels.Set{"foo": "bar"}),
            }
            // Specific selectors per type of object
            opts.SelectorsByObject[&corev1.Secret{}] = cache.ObjectSelector{
                Label: labels.SelectorFromSet(labels.Set{"foo": "bar"}),
            }
            return cache.New(conf, opts)
        },
    })
    if err != nil {
        return err
    }
    r := &InvReconciler{}
    if err := r.SetupWithManager(mgr); err != nil {
        return err
    }