RabbitMQ: Bi-directional federated brokers, how do you make all queues pop the message?

Question

I have two brokers configured[1] with federation plugin. Both are pointing to each other as upstream.

My test is:

publish a message on broker A
consume on broker B

The result is:

consuming on broker B works
< good > the queue on broker B pops the message
< not good > the queue on broker A still has the message
- < reason why this is not good > The issue I see is: if I always publish on one broker and then always consume on the other --> then the queue on the publishing broker will grow until it's full and start dropping messages.

The result I would like is:

both queues on broker A and B pop their messages when the consumer consumes on broker B

How do I configure RabbitMQ to pop the message from all queues when a consumer consumes the message on broker B? Right now I am trying to do so with RabbitMQ Federation plugin.

[1] The two brokers point to each other as upstreams and I configure them the same way as described in the "simple example" given by the documentation except that there are two brokers each pointing to each other as upstream. The code for the publisher looks like this and the code for the consumer looks like this.

I suppose you are using queue federation right? and you want to consume messages from one broker and also the other broker have to be aligned, right? — Gabriele Santomaggio, Nov 10 '15 at 07:59
@Gabriele re - "[are] you using queue federation?": I am using the ["simple example"](https://www.rabbitmq.com/federation.html) from the Federation "Getting Started" section. I think this "simple example" uses Federated Exchanges. (Also my RabbitMQ is version 3.1.5 which does not have Federated Queues ([Federated Queues are available for versions 3.2 and onward](https://www.rabbitmq.com/federated-queues.html)). — Trevor Boyd Smith, Nov 10 '15 at 13:56
@Gabriele are you saying that federated-queues may be the solution to the problem? (I ask because I noticed you are much more experienced (author of RabbitMQ Cookbook which I bought already)). ||| From ten minutes [reading about federated queues](https://blog.pivotal.io/pivotal/news-2/new-release-rabbitmq-3-2-0-with-federated-queues), it sounds like they would do exactly what I need/want (i.e. a consumer on a different broker would consume from a federated-queue and that would cause the message to be consumed on the upstream-queue). Does my explanation sound right to you? — Trevor Boyd Smith, Nov 11 '15 at 20:15
have you considered to use the cluster feature instead of federation? — Gabriele Santomaggio, Nov 12 '15 at 07:08
I'm using federation instead of clustering because federation is meant for WANPHY or unreliable links. — Trevor Boyd Smith, Nov 12 '15 at 11:21
Good choose, let me know if it is ok! I am not always present on SO. — Gabriele Santomaggio, Nov 12 '15 at 11:26

Yuci · Answer 1 · 2020-04-25T14:39:24.097

@Trevor Boyd Smith, probably option 2 or 3 as shown below are something you could consider.

Option 1: Bidirectional federated exchange

A message will end up in the both broker A and B, one copy each broker, independent of each other. In other words, e.g., even after broker B has delivered the message to its consumer, the other copy of the message still remains in broker A.

Advantage: You will always have two copies of the message, one in each broker, which is highly available.

Disadvantage: You need to have a consumer connected to each broker.

Option 2: Bidirectional federated queue

A message will end up in one of the two brokers. By default, the broker where the message has been published will have the priority to enqueue the message; however if only the other broker has got a consumer, the message will move to the other broker.

It does not matter which broker the message ends up in, the message will be delivered once and once only by a consumer connected to either of the brokers.

Advantage: The message will be delivered once and once only to a consumer connected to either of the brokers.

Disadvantage: There will be only one copy of the message. If the broker that has got the message goes down, the other broker can not get the message. But if you are fine with eventual consistency, this option is OK. The reason is when the problematic broker comes back up running, the message would be available, eventually.

Option 3: Bidirectional federated exchange and queue

In this case, a message will end up in both brokers, one copy each broker. The same message will be delivered to a consumer connected to either of the brokers, twice! Once the message has been delivered twice, it will be gone in both brokers. (If there are two consumers, one connected to each broker, each broker will deliver the same message to its consumer once.)

Advantage: The consumer can connect to either of the brokers, and the message in each broker will be delivered and dequeued.

Disadvantage: The same message will be delivered twice. The workaround would be, before handling the message, check if the same message has already handled.

Note:

It does not mean which option is better than the other one or ones. It all depends on your use case, and there are many other configurations that can come into play, which could change the behaviour.

score 2 · Answer 2 · answered Nov 12 '15 at 08:33

I created this environment:

Server A, Server B.

Created a bidirectional federated in this way:

Federation Upstream: Server_B = amqp://servera

Federation Upstream: Server_A = amqp://serverb

Then created the same policy on both the servers:

Pattern : ^fed\.
Apply to: all   
federation-upstream-set:all

Created one queue to the Server A called: fed.test1 then created a consumer to the Server B as:

 ConnectionFactory factory = new ConnectionFactory();
 factory.setHost("localhost");
 factory.setPort(5673);
 Connection connection = factory.newConnection();
 Channel channel = connection.createChannel();


Consumer consumer = new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body)
                        throws IOException {
String message = new String(body, "UTF-8");
System.out.println(" [x] Message   '" + message );
                }
            };
channel.basicConsume("fed.test1",  true, consumer);

Then published a message to the Server A ---> fed.test1

The message has been consumed to the Server B and the queues messages count is zero to both the queues (Server A, Server B).

This works as you expected.

Hope it helps.

Note that this does not maintain a single logical queue across the federation. The messages on Server A are sent to Server B only when there is a consumer handling messages. In other words, this does not create a redundant instance of the messages on each server. Exchange federation, the other option, will simply copy the messages to the downstream server leaving two separate but identical consumable messages on each server. There does not appear to be an option that will maintain a copy of the messages and ensure single delivery. — Sean Anderson, Feb 09 '18 at 05:56

RabbitMQ: Bi-directional federated brokers, how do you make all queues pop the message?

2 Answers2

Option 1: Bidirectional federated exchange

Option 2: Bidirectional federated queue

Option 3: Bidirectional federated exchange and queue