k8s controller watch other controller CR

Question

I’ve k8s operator which works as expected, I need to add a “watch” to other operator CRD (not mine), to make it simple lets call it extCR and our operator cr called inCR,

I tried the following but there is an issue how its right to trigger the reconcile.

func (r *Insiconciler) SetupWithManager(mgr ctrl.Manager) error {
                return ctrl.NewControllerManagedBy(mgr).
                    For(&Inv1alpha1.Iget{}}).
                    Watches(&source.Kind{Type: &ext.Se{}},  handler.EnqueueRequestsFromMapFunc(r.FWatch)).
                    Complete(r)
}
    
func (r *Insiconciler) FWatch(c client.Object) []reconcile.Request {
        val := c.(*ivi.Srv)
        req := reconcile.Request{NamespacedName: types.NamespacedName{Name: val.Name, Namespace: val.Namespace}}
        return []reconcile.Request{req}
}

The problem here that I trigger the reconcile with the extCR , I want inside the FWatch to update the inCR and start the reconcile with inCR and not with extCR, how can I do it ?

I mean, to avoid something like the following code as sometimes the reconcile is done for the inCR and sometimes for the extCR and than I can get some ugly if's

func (r *Insiconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
        var inCR FOO
        var extCR BAR
    
        if err := r.Get(ctx, req.NamespacedName, &inCR); err != nil {
            return ctrl.Result{}, err
        }
        
        if err := r.Get(ctx, req.NamespacedName, &extCR); err != nil {
            return ctrl.Result{}, err
        }

I want to know what is the right/clean way to handle such case

case when you need to listen to externalCR (not part of your controller) and also internalCR (from your controller) .

One more thing - the CR are different GVK but the exteranlCR contain lot of fields which is not required, just some of them. but the required fields is having the same names on both cr's

update

type inCR struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`

    Spec   inSpec  `json:"spec,omitempty"`  / / ————————here is the difference 
    Status InsightTargetStatus `json:"status,omitempty"`
}

//————— This is defined on other program which is not owned by us, therefore cannot “reuse”

type Bar struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`

    Spec   extSpec  `json:"spec,omitempty"`    // ———————here is the difference 
    Status ServiceStatus `json:"status,omitempty"`
}

And inSpec is having the following fields (subset of extSpec)

type inSpec struct {
    name string
    age  int
}

and extSpec have those fields and many more which is not related

type extSpec struct {
    name string   
    age  int
    foo string  // not relevant
    bar string  // not relevant
    bazz string // not relevant
}

at the end, Inside the reconcile I need to move the relevant fields to some functions. exactly same functions just take sometime the fields from extCR and sometimes for inCR, according to the event that happens (like updating the extCR or update the inCR by users )

Update2

func sharedLogic(r reconciler, ctx context.Context, c client.Object) (ctrl.Result, error) {
    
    
    in := c.(*inCR)
    
    
    vPass , e := vps.Get(ctx, r.Client, in.Spec.foo, in.Spec.bar)
    
    
     return ctrl.Result{}, nil
    }

But for extCR I should do the following


func sharedLogic(r reconciler, ctx context.Context, c client.Object) (ctrl.Result, error) {


ext := c.(*extCR)


vPass , e := vps.Get(ctx, r.Client, ext.Spec.val.foo, ext.Spec.val.bar)


 return ctrl.Result{}, nil
}

Answer 1

Few things to keep in mind:

• Each controller is responsible for exactly one resource.
• Reconcile request contains the information necessary to reconcile a Kubernetes object. This includes the information to uniquely identify the object - its Name and Namespace. It does NOT contain information about any specific Event or the object contents itself.

---

You can create a second controller without the resource definition. In your main file, both controllers will be registered.

This could be useful if the CRDs are not related at all or if the external resource references the internal one, so you can make changes to the internal resource in the external reconciler.

kubebuilder create api --group other --version v2 --kind External \
 --resource=false --controller=true

This gives you a controller with a SetupWithManager method that looks like the below.

func (r *ExternalReconciler) SetupWithManager(mgr ctrl.Manager) error {
    return ctrl.NewControllerManagedBy(mgr).
        // Uncomment the following line adding a pointer to an instance of the controlled resource as an argument
        // For().
        Complete(r)
}

Note how the For method is commented out because you need to import the resource to watch from somewhere else and reference it.

import (
    ...
    otherv2 "other.io/external/api/v2"
)
...
func (r *ExternalReconciler) SetupWithManager(mgr ctrl.Manager) error {
    return ctrl.NewControllerManagedBy(mgr).
        For(&otherv2.External{}).
        Complete(r)
}

If you cannot import the external resource you could fall back to mocking it yourself but this is probably not a very clean way. You should really try to import it from the other controller project.

kubebuilder edit --multigroup=true
kubebuilder create api --group=other --version v2 --kind External \
  --resource --controller

---

Another way is when the resources are related to each other such that the internal resource has a reference in its spec to the external resource and knows how to get the external resource in its spec, when it reconciles. An example of this can be found here https://book.kubebuilder.io/reference/watching-resources/externally-managed.html

type InternalSpec struct {
    // Name of an external resource
    ExternalResource string `json:"externalResource,omitempty"`
}

This means that in each reconciliation loop, the controller will look up the external resource and use it to manage the internal resource.

func (r *InternalReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    _ = log.FromContext(ctx)

    internal := examplev1.Internal{}
    if err := r.Get(context.TODO(), types.NamespacedName{
        Name:      req.Name,
        Namespace: req.Namespace,
    }, &internal); err != nil {
        return ctrl.Result{}, err
    }

    external := otherv2.External{}
    if err := r.Get(context.TODO(), types.NamespacedName{
        // note how the name is taken from the internal spec
        Name:      internal.Spec.ExternalResource,
        Namespace: req.Namespace,
    }, &internal); err != nil {
        return ctrl.Result{}, err
    }

    // do something with internal and external here

    return ctrl.Result{}, nil
}

The problem with this is, that when the internal resource does not change, no reconciliation event will be triggered, even when the external resource has changed. To work around that, we can trigger the reconciliation by watching the external resource. Note the Watches method:

func (r *InternalReconciler) SetupWithManager(mgr ctrl.Manager) error {
    return ctrl.NewControllerManagedBy(mgr).
        For(&examplev1.Main{}).
        Watches(
            &source.Kind{Type: &otherv2.ExternalResource{}},
            handler.EnqueueRequestsFromMapFunc(r.triggerReconcileBecauseExternalHasChanged),
            builder.WithPredicates(predicate.ResourceVersionChangedPredicate{}),
        ).
        Complete(r)
}

In order to know for which internal object we should trigger an event, we use a mapping function to look up all the internal that have a reference to the external resource.

func (r *InternalReconciler) triggerReconcileBecauseExternalHasChanged(o client.Object) []reconcile.Request {
    usedByInternals := &examplev1.InternalList{}
    listOps := &client.ListOptions{
        FieldSelector: fields.OneTermEqualSelector(".spec.ExternalResource", o.GetName()),
        Namespace:     o.GetNamespace(),
    }
    err := r.List(context.TODO(), usedByInternals, listOps)
    if err != nil {
        return []reconcile.Request{}
    }
    requests := make([]reconcile.Request, len(usedByInternals.Items))
    for i, item := range usedByInternals.Items {
        requests[i] = reconcile.Request{
            NamespacedName: types.NamespacedName{
                Name:      item.GetName(),
                Namespace: item.GetNamespace(),
            },
        }
    }
    return requests
}

---

Since you updated your question, I suggest doing something like below.

I am creating a new project and 2 controllers. Note on the second controller command no resource is created along with the controller. this is because the controller will watch an external resource.

mkdir demo && cd demo
go mod init example.io/demo
kubebuilder init --domain example.io --repo example.io/demo --plugins=go/v4-alpha
kubebuilder create api --group=demo --version v1 --kind Internal --controller --resource
kubebuilder create api --group=other --version v2 --kind External --controller --resource=false

$ tree controllers
controllers
├── external_controller.go
├── internal_controller.go
└── suite_test.go

Now we need some shared logic, for example by adding this to the controllers package. We will call this from both reconcilers.

// the interface may need tweaking
// depending on what you want to do with
// the reconiler
type reconciler interface {
 client.Reader
 client.Writer
 client.StatusClient
}

func sharedLogic(r reconciler, kobj *demov1.Internal) (ctrl.Result, error) {
 // do your shared logic here operating on the internal object struct
 // this works out because the external controller will call this passing the
 // internal object
 return ctrl.Result{}, nil
}

Here is an example for the internal reconciler.

func (r *InternalReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
 _ = log.FromContext(ctx)
 obj := demov1.Internal{}
 if err := r.Get(ctx, req.NamespacedName, &obj); err != nil {
  return ctrl.Result{}, err
 }
 return sharedLogic(r, &obj)
}

And in the external reconciler we do the same.

func (r *ExternalReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
 _ = log.FromContext(ctx)
 // note, we can use the internal object here as long as the external object
 // does contain the same fields we want. That means when unmarshalling the extra
 // fields are dropped. If this cannot be done, you could first unmarshal into the external
 // resource and then assign the fields you need to the internal one, before passing it down
 obj := demov1.Internal{}
 if err := r.Get(ctx, req.NamespacedName, &obj); err != nil {
  return ctrl.Result{}, err
 }
 return sharedLogic(r, &obj)
}

func (r *ExternalReconciler) SetupWithManager(mgr ctrl.Manager) error {
 return ctrl.NewControllerManagedBy(mgr).
 // note the external resource is imported from another project
 // you may be able to watch this without import by creating a minimal
 // type with the right GKV
  For(otherv2.External{}).
  Complete(r)
}