3

I'm building a system in which I'd like to use RabbitMQ as the main message passing mechanism.

My system requires very strong guarantees for message delivery, as in, messages must arrive at their destination and acknowledged, otherwise I expect them to be requeued, and redelievered.

The use case is maintaining eventual consistency between 2 systems

I'd also like to avoid having to manage any state regarding the messages in my application.

I know that publishing a persistent message does not provide a 100% guarantee that it was persisted to disk, because fsync(2) is not called for every message.

thus, my question is - Can I use RabbitMQ as a 100% reliable* message passing mechanism?

I've read about transactions, but couldn't find anything besides a 2011 blogpost which also doesn't explicitly state that an fsync(2) is performed for every message, or that a (synchronous) failure occurs if it doesn't succeed.

* by reliable I mean that the messages should eventually be delivered, always. duplicate messages are acceptable.

nadavvadan
  • 3,930
  • 1
  • 17
  • 29
  • I think this is too broad for Stack Overflow. The docs here on [reliable delivery](https://www.rabbitmq.com/reliability.html) refer you to the mailing list. My personal opinion is _yes_, RabbitMQ gives you reliable delivery but someone else might cosider x, y or z failure condition as unacceptable. – Adam Sep 04 '18 at 12:32
  • Thanks for the comment, @Horba, but I feel that this question does deserve an answer, and is not too broad. I’m not talking about general reliability, but instead describe a specific situation in which delivery failures are possible, and request a solution for this specific situation. – nadavvadan Sep 04 '18 at 12:43

1 Answers1

1

Can I use RabbitMQ as a 100% reliable message passing mechanism?

Yes, use persistent messages, publisher confirms, and consumer acknowledgements - link.

When your publisher receives a confirmation, the message has been written and synced to disk.

NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.

Luke Bakken
  • 8,993
  • 2
  • 20
  • 33
  • I’ve read about publisher confirms briefly and did not understand that they provide a stronger guarantee than persistent messages on durable queues. This covers most of my concerns, except for managing state in my application. Anyway, I guess you can’t have it all. Thanks! – nadavvadan Sep 04 '18 at 14:38
  • Publisher confirms and persistent messages are two separate things. You can have non-persistent messages with publisher confirms, and vice versa. It is the combination of using both that is most reliable. Yes, you can have it all. – Luke Bakken Sep 04 '18 at 22:43
  • Yes, that is very clear to me. I still have to manage the state of failed messages though - hence I can't, technically, "have it all". – nadavvadan Sep 05 '18 at 07:02