Subscribe to all tenants using a wildcard in Eclipse-hono

Question

I'm trying to consume data from hono. I do so by following the guide on Starting a consumer on hono documentation.
I'm currently trying to subscribe to all tenants by add --tenant.id=* at the end of the the mvn command. This results in following command:

mvn spring-boot:run -Drun.arguments=--hono.client.host=localhost,--hono.client.username=consumer@HONO,--hono.client.password=verysecret,--destination.TopicTemplate=gw/\!{tenant}/\!{device}/alp,--destination.Hoscalhost,--destination.Port=11883,--tenant.id=*

I'm not getting any messages when I subscribe like this. When I subscribe using the example command (only for DEFAULT_TENANT), I'm consuming the messages.
The current user permission looks like this:

"consumer@HONO": {
      "mechanism": "PLAIN",
      "password": "verysecret",
      "authorities": [ "application" ]
    }

The current application role looks like this:

"application": [
  {
    "resource": "telemetry/*",
    "activities": [ "READ" ]
  },
  {
    "resource": "event/*",
    "activities": [ "READ" ]
  },
  {
    "resource": "control/*",
    "activities": [ "READ", "WRITE" ]
  }

Both of them are still the original ones from Hono github.

EDIT: The consumer also subscribes to event/tenant. In my case this is event/. Events published on topic event/DEFAULT_TENANT and event/MY_TENANT are consumed. However, the consumer for telemetry/ seems not to be registered.

score 1 · Accepted Answer · answered Oct 26 '18 at 11:11

1

I've finally found out what was going on.
It seems the message is blocked in the QPID dispatch router because of folowwing error: "Parse tree match not found".
This can be resolved by changing the qpid configuration. In this configuration you should be able to find following records:

["linkRoute", {
    "prefix": "event/",
    "direction": "in",
    "connection": "broker"
  }],

  ["linkRoute", {
    "prefix": "event/",
    "direction": "out",
    "connection": "broker"
  }],

  ["address", {
    "prefix": "telemetry/",
    "distribution": "balanced"
  }],

It creates linkroutes (in and out) for event topic but not for the telemetry topic. Adding these records for the telemetry topic resolves the problem.

["linkRoute", {
    "prefix": "event/",
    "direction": "in",
    "connection": "broker"
  }],

  ["linkRoute", {
    "prefix": "event/",
    "direction": "out",
    "connection": "broker"
  }],

  ["linkRoute", {
    "prefix": "telemetry/",
    "direction": "in",
    "connection": "broker"
  }],

  ["linkRoute", {
    "prefix": "telemetry/",
    "direction": "out",
    "connection": "broker"
  }],

  ["address", {
    "prefix": "telemetry/",
    "distribution": "balanced"
  }],

answered Oct 26 '18 at 11:11

Bob Claerhout

781
5
24

How did you do this? I am running Hono through the docker swam deploy scritp, how can I add this options? – mariolpantunes May 17 '19 at 20:59
Hi @mariolpantunes , The file mentioned to change is located here: hono/deploy/src/main/config/qpid/qdrouterd-with-broker.json. On github, this is following file: https://github.com/eclipse/hono/blob/master/deploy/src/main/config/qpid/qdrouterd-with-broker.json. However, as Kai mentioned, it is not intended to subscribe to all tenants (as I have learned by now). What are you trying to achieve? Maybe this can be fixed properly by some other means. – Bob Claerhout May 21 '19 at 15:53
This is an academic work. We have a simple bridge that requests all events, telemetry and control messages and store them into a database. – mariolpantunes May 21 '19 at 17:47
Ok, maybe [ditto](https://www.eclipse.org/ditto/) can help you out. This [tutorial](https://www.eclipse.org/ditto/2018-05-02-connecting-ditto-hono.html) describes how you can connect ditto and hono. Though you'll have to create a connection for each tenant (is equal to subscribing to all different tenants). After that, you ca update a feature to which you can subscribe. If you need any more help on that, maybe you can create a ticket for that. – Bob Claerhout May 22 '19 at 07:22
The bridge will register devices into ditto automatically. So the proposed solution is not possible in our use case. – mariolpantunes May 22 '19 at 16:07
As Kai already mentioned, the idea of a tenant is to isolate it. Maybe you can help us understand why you want to use different tenants in the first place. – ctron May 23 '19 at 08:56

score 1 · Answer 2 · answered Oct 29 '18 at 08:55

Hono does not (as of now) support consuming messages of all tenants. The consumer is always scoped to a single tenant only. This is also reflected in the (northbound) Telemetry and Event API specifications.

The usage of wildcard characters in order to receive data for multiple/all tenants is not supported. The change you have made to the Dispatch Router configuration may have led you to believe that it does work indeed. However, defining the telemetry address to use link routing instead of the default message routing has some consequences you should be aware of:

All telemetry messages will be routed to the message broker (Artemis) instead of being routed directly to consumers attached to the Dispatch Router. This means that all messages will be written to a queue/topic in Artemis. Depending on the Artemis configuration this might also mean that (telemetry) messages get persisted which will have quite a negative impact on throughput.
Your clients/consumers will now explicitly depend on the (Artemis) broker's support for wildcards being used in AMQP 1.0 link source addresses to receive messages from multiple addresses. While this might be what you want to achieve in the first place, beware that it ties your application to the specific implementation of the AMQP Messaging Network (in this case Artemis) which is not part of Hono.

In our case, we have an existing backend application. These are micro services which are responsible for processing (parsing, calculating application specific parameters, ...) all incoming messages. Parsing (e.g.) is the same for a specific message type, whether this message originates from tenant A or tenant B. As suggested [here](https://github.com/eclipse/hono/issues/866) I can use Apache Camel to forward all messages from a specific tenant to a generic processor/parser. However, if I'm correct, I will have to configure each specific tenant. As you can imagine, we would like to avoid this. — Bob Claerhout, Nov 05 '18 at 10:59

Subscribe to all tenants using a wildcard in Eclipse-hono

2 Answers2