4

I'm currently using RabbitMQ (bunny) at VersionEye to import meta information about GitHub repos via the GitHub API. Depending on the number of the GitHub repos a task can take a view seconds up to a couple minutes. I create a new message/task like this: 

queue.publish( msg, :persistent => true )

One of the workers will get the message and perform the work. The worker sets his status (RUNNING, DONE) in Memcached. That way I know when a task is done!

I would like to get rid of Memcached in that process. I would like to get a status to a msg from RabbitMQ. Something like this would be ideal: 

status = queue.publish( msg, :persistent => true )
status = queue.status( msg )

Unfortunately I couldn't find anything like that in the RabbitMQ or Bunny docu. Does anybody know how to get the status of a message from RabbitMQ?

Robert Reiz
  • 4,243
  • 2
  • 30
  • 43

1 Answers1

2

There are no such feature to get specific message status in RabbitMQ out of the box.

Alternatively to Memcache-based solution to track message status you can use RabbitMQ itself - declare message-specific queue in application and track it status. The whole concept is very close to Remote procedure call (RPC). Say, if it doesn't exists - message still in queue, if it exists - check it content (send "message received" message from message consumer when it was received and "message processed" when message will be processed). The biggest cons is that you queues takes some resources - mainly, memory, which matters if you have huge message flow. Also you will have to cleanup response queues (may be done with queue ttl extension - don't be confused with per-queue message ttl).

To use RabbitMQ in a way described above first make sure you really need this (maybe old-school RPC pattern is what you want, + maybe add some message ttl + dead lettering support to limit response time to reasonable time). If you sure, it is always good idea to move tons of queues to separate vhost for ease of maintenance the reset of queues. Also make sure you will not exceed server resources limit.

pinepain
  • 12,453
  • 3
  • 60
  • 65
  • 1
    OK. Good to know that the desired feature doesn't exist. In that case I just keep going with the current Memcached implementation, because it works already :-) – Robert Reiz Sep 14 '14 at 17:39