1

I have scenario for which I am looking for a message queue service which supports below:

  1. Ease to Use
  2. Very high in performance
  3. Message once read shouldn't be available for other consumers.
  4. Should have capability to delete the message once read.
  5. Message once published should not get dropped.

The scenario which I have is described below:

  1. There are many publishers.
  2. There will be many consumers.
  3. Queuing server and consumers residing on same machine, but publishers are residing on different machines.

Please let me know best queuing service apart from Rabbitmq and sqs satisfying above points

mohit3081989
  • 441
  • 6
  • 13
  • I tried Rabbitmq thinking it is appreciated in each forum. But the performance wise I highly doubt, it will scale. It is taking around 3ms to write a single message to local rabbitmq server. – mohit3081989 Oct 22 '13 at 18:24

3 Answers3

4

I would recommend Apache Kafka: http://kafka.apache.org/

If you want to know a comparison between Kafka and RabbitMQ you should read this article: http://www.quora.com/RabbitMQ/RabbitMQ-vs-Kafka-which-one-for-durable-messaging-with-good-query-features

Also, you should take a look to this: ActiveMQ or RabbitMQ or ZeroMQ or

Community
  • 1
  • 1
hveiga
  • 6,725
  • 7
  • 54
  • 78
0

Kafka as far I know is mainly meant for real data propagation and I think my requirement doesn't require something like kafka. I have used SQS but the only problem I have with SQS is high latency. Publisher pushed message to queue and consumer keep on polling for new message, this implementation is hitting me with very high latency. My requirement is simple as follows:

  1. Queue service should have high availability and reliability like SQS
  2. Latency should be very high lets say not more than 10ms. (here 10ms includes publishing and receiving the message).
  3. Also my message size is very small say not more than 20-30 bytes.

I have thought of using redis, in which I will be pushing the messages to a list and workers will keep on popping them back to back till list becomes empty, but I have not done any benchmarking on that. So here I really need suggestion so I go in right direction.

Thanks,

mohit3081989
  • 441
  • 6
  • 13
0

For some of my system integration projects I met the MQ-tasks. Several rich costumers wants the production solutions like IBM WebSphere MQ, but I think it's too much expressive and difficult.

I found and used the simple and stable analog: e-mail server.

All integrated systems got the local e-mail boxes. Messages are e-mail, with command-code in subject and json in attachments. E-mail server listen and dispatch all queues, to recipients or to groups of them. E-mail protocols are stable and all developers know a lot of tools to work with it. Sysadmins and testers use the simple e-mail clients for testing and auditing. All e-mail servers have a logging tools.

It's best and easy solution, and I suggest it for most of integration projects.

Y.N
  • 4,989
  • 7
  • 34
  • 61