0

What is the best way to identify the origin of a (bad) AMQP message in rabbitmq?

The scenario is several clients posts messages to a server which if successful result in messages posted to a fanout exchange (for pub/sub use). The client is also a subscriber to this queue and does not have a reply queue of its own.

If the server decides a message is invalid it nacks and discards it.

There are two additional things we want:

  • to publish a message which either the client or a monitoring system might be used to say "stop sending me garbage"
  • to log diagnostics about bad messages

What is an appropriate strategy for this?

Which header fields are appropriate for identifying the originator of a message?

Are they any other considerations?

The appid header field can identify the sending application but it is not sufficient by itself as there could be multiple instances on one or more machines.

A bad client could perhaps use the messageid to identify messages it sent (can this be assigned or accessed by the client).

A monitoring system (e.g. prometheus+kubernetes) would need more to try and kill the bad client process for example.

I'm assuming an appropriate approach is to require clients to set certain header fields.

Secondary to that what can we do for clients that fail don't meet our requirements there?

Bruce Adams
  • 4,953
  • 4
  • 48
  • 111

1 Answers1

1

I have limited experience with AMQP so please improve on this if you can

It is sufficient to use host+pid to identify a process

The appid is free form so one approach might be to include more than just the application Id in it. e.g.

appid: applicationid.serverip.pid

This might seem like stretching the semantics of "app id" but app_id is entirely application defined. An application can identify itself with the "app id" with a simple string comparison.

arbitrary key values are permitted on top of the basic properties so you could instead/as well add properties like:

origin-host: serverip
origin-pid: pid

This may require a small additional programming effort over using 'builtin' fields like app_id. There is also a small 'over the wire' cost associated with this (custom properties are serialized as positional arguments in message header instead of associative array to save bandwidth). There may be a trade off with this versus decoding the string in the appid field.

You could also set up dead letter exchanges to hold the messages themselves but the messages would require operator intervention to make use of. So I think this would only help with diagnostics.

Bruce Adams
  • 4,953
  • 4
  • 48
  • 111