1

I am new in Spring-Integration, and lloking to the solution of the following task.

We try to create an MQTT-Consumer ( based on the mqtt inbound channel adapter), which recieves messages on 7 topics from up to 50 publishers.

As the adapter processes the messages sequentially, one has to scale parallelize the processing, or gets the rest of the messages dropped by the broker.

First idea for scaling was a single adapter per topic, with corresponding converters. It consumes well messages from one publisher, but can not withstend so many publishers, with the queued or direct outbound-channel.

Currently thinking about using the executor channel with the number of message handlers(own converters) for each topic. Those afterwards will pipe their results into the corresponding queued channels, that would allow to process and propagate the messages in batches.

However, I am not sure whether creating a single single adapter per topic is a good idea, and am searching for any suggestions about the architecture improvements. Thanks in advance!

JustToAsk
  • 11
  • 1
  • An adapter per topic is fine to add some parallelism; if the consumers still can't keep up, you are in trouble, unless you don't mind losing messages. If you use an executor or queue channel to hand off to another thread, the message will be considered delivered by Paho (according to the QOS setting). Such messages won't be redelivered after a failure. – Gary Russell Nov 29 '18 at 18:37

0 Answers0