How to Connect knative-eventing to RabbitMQ External Source

Question

I am trying to connect Knative with an existing RabbitMQ queue outside the Kubernetes cluster, but after testing and looking at the documentation I am not sure this can be done, does anyone know how to use Knative-eventing with an external RabbitMQ as a message source ?

Regards!

score 1 · Answer 1 · answered May 19 '22 at 10:27

1

As listed in the Knative Eventing sources catalog, there is a RabbitMQ source being developed and maintained by the community (largely contributors from VMware and the RabbitMQ team).

Documentation and install instructions are here; in the case of bugs, please report them in the associated repo.

answered May 19 '22 at 10:27

E. Anderson

3,405
1
16
19

Hi! But in the documentation you give me it says: "Before we can create the Knative Eventing Source, we first need to create a RabbitMQ Cluster:" Besides referring to the creation of a broker, it implies that it only works with RabbitMQ clusters within the Kubernetes cluster itself and I want to use one already available by passing the connection data. – Juan May 20 '22 at 06:13
Hey @Juan, The source should be able to connect to an arbitrary instance of Rabbit. Here, https://github.com/knative-sandbox/eventing-rabbitmq/blob/main/test/e2e/config/source/rabbitmq-source.yaml is the e2e example where the Source is connecting to a Rabbit Instance using connection data (the instance just happens to be instantiated using the RMQ Cluster Operator for ease of use). – sameer May 23 '22 at 13:56
Which version of Knative are you using ? The team is working towards an MVP of the integration. I created an issue to clarify the instructions here - https://github.com/knative-sandbox/eventing-rabbitmq/issues/783 – sameer May 23 '22 at 13:57
Hi @sameer I replied into another answer because in this box field has a character limits – Juan May 24 '22 at 11:53

Gabriel Freites · Answer 2 · 2022-05-24T22:51:19.793

1

Hey @Juan you were absolutely right about our docs, here I'll leave a PR fixing part of them while it's being reviewed and merged, and with a clear example about using external RabbitMQ instances: https://github.com/knative-sandbox/eventing-rabbitmq/pull/786/files#diff-4fdb9e4eb3a1c9da58e4445d94aa5ce4573b5c8d005f20c41c767b07c09a2418

Hope this helps =), and thanks for the feedback! If you find anything wrong, you can comment on the PR or leave a reply over here!

edited May 24 '22 at 22:51

answered May 24 '22 at 22:49

Gabriel Freites

11
1
2

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community May 28 '22 at 16:55

score 0 · Answer 3 · answered May 24 '22 at 11:51

thank you @sameer for your answer, I think I am not understanding you correctly.

After visiting the link you give me, I have launched:

kubectl apply --filename https://github.com/knative-sandbox/eventing-rabbitmq/releases/latest/download/rabbitmq-source.yaml

To install the rabbitmq source and generated the secret:

kubectl create secret generic rabbitmqc-default-user -n knative-eventing --from-literal=user=root --from-file=password=/tmp/password

And the following resource:

apiVersion: sources.knative.dev/v1alpha1
kind: RabbitmqSource
metadata:
  name: rabbitmq-source
  namespace: knative-eventing
spec:
  broker: "host.external.dns:5671/"
  connectionSecret:
    name: "rabbitmqc-default-user"
  user:
    secretKeyRef:
      name: "rabbitmqc-default-user"
      key: "username"
  password:
    secretKeyRef:
      name: "rabbitmqc-default-user"
      key: "password"
  exchangeConfig:
    name: "logs"
    type: "fanout"
    durable: true
    autoDelete: false
  queueConfig:
    name: "test"
    routingKey: ""
    durable: false
    autoDelete: false
  channelConfig:
    parallelism: 10
  sink:
    ref:
      apiVersion: v1
      kind: Service
      name: recorder

But after generating the resources I get these errors:

knative-sources/rabbitmq-controller-manager

│ W0524 11:45:04.108665       1 reflector.go:324] k8s.io/client-go@v0.23.5/tools/cache/reflector.go:167: failed to list *v1beta1.Binding: the server could not find the requested resource (get bindings.rabbitmq.com)                                                                   │
│ E0524 11:45:04.108718       1 reflector.go:138] k8s.io/client-go@v0.23.5/tools/cache/reflector.go:167: Failed to watch *v1beta1.Binding: failed to list *v1beta1.Binding: the server could not find the requested resource (get bindings.rabbitmq.com)                                 │
│ W0524 11:45:04.109915       1 reflector.go:324] k8s.io/client-go@v0.23.5/tools/cache/reflector.go:167: failed to list *v1beta1.Queue: the server could not find the requested resource (get queues.rabbitmq.com)                                                                       │
│ E0524 11:45:04.109935       1 reflector.go:138] k8s.io/client-go@v0.23.5/tools/cache/reflector.go:167: Failed to watch *v1beta1.Queue: failed to list *v1beta1.Queue: the server could not find the requested resource (get queues.rabbitmq.com)                                       │
│ W0524 11:45:04.109976       1 reflector.go:324] k8s.io/client-go@v0.23.5/tools/cache/reflector.go:167: failed to list *v1beta1.Exchange: the server could not find the requested resource (get exchanges.rabbitmq.com)                                                                 │
│ E0524 11:45:04.109985       1 reflector.go:138] k8s.io/client-go@v0.23.5/tools/cache/reflector.go:167: Failed to watch *v1beta1.Exchange: failed to list *v1beta1.Exchange: the server could not find the requested resource (get exchanges.rabbitmq.com)                              │
│ W0524 11:45:05.253806       1 reflector.go:324] k8s.io/client-go@v0.23.5/tools/cache/reflector.go:167: failed to list *v1beta1.Exchange: the server could not find the requested resource (get exchanges.rabbitmq.com)                                                                 │
│ E0524 11:45:05.254035       1 reflector.go:138] k8s.io/client-go@v0.23.5/tools/cache/reflector.go:167: Failed to watch *v1beta1.Exchange: failed to list *v1beta1.Exchange: the server could not find the requested resource (get exchanges.rabbitmq.com)                              │
│ W0524 11:45:05.443628       1 reflector.go:324] k8s.io/client-go@v0.23.5/tools/cache/reflector.go:167: failed to list *v1beta1.Queue: the server could not find the requested resource (get queues.rabbitmq.com)

The truth is that I have searched and searched and I have not found information on how to connect it to an external RabbitMQ to Kubernetes, maybe I am not understanding the documentation well.

Elad Hirsch · Answer 4 · 2023-05-06T12:22:52.023

This error message indicates that xxx.rabbitmq.com CRD is not installed in the current k8s cluster.

│ W0524 11:45:04.109915 1 reflector.go:324] k8s.io/client-go@v0.23.5/tools/cache/reflector.go:167: failed to list *v1beta1.Queue: the server could not find the requested resource (get queues.rabbitmq.com) │

You need to install the RabbitQA messaging-topology-operator in order to add the missing CRDS

as mentioned in the docs:

Note: An external RabbitMQ instance can be used, but if you want to use the Source without predeclared resources (specifically the Exchange and Queue), the RabbitMQ Message Topology Operator needs to be installed in the same Kubernetes Cluster as the Source.

follow this command :

kubectl apply -f https://github.com/rabbitmq/messaging-topology-operator/releases/latest/download/messaging-topology-operator-with-certmanager.yaml

How to Connect knative-eventing to RabbitMQ External Source

4 Answers4