Eventbridge - Use FIFO SQS for deduplication

Question

I need some events to be delivered exactly once, but I have no control of the message processor (so I can‘t make the recipient idempotent).

Is it possible to route events from Eventbridge to a FIFO SQS for deduplication and from the FIFO sqs to the recipient (lambda on other account? Would this achieve exact-once delivery?

score 1 · Answer 1 · answered Apr 08 '22 at 21:39

1

Can you set dynamically messageGroupId from the payload content when you set SQS FIFO as a target for your EventBridge rule?

From the current setup I see that it can be only a hardcoded value.

answered Apr 08 '22 at 21:39

Julian Dimitrov

31
2

As far as I can tell this is not possible, I have just hit the same problem, if you have found a solution I'd love to know what it was? – Martin W May 13 '22 at 10:38

score 0 · Accepted Answer · answered Feb 08 '22 at 23:37

0

It EventBridge (EB) has at-least-once deliver which means you can get more then one event of the same type. But if this is not an issue, and your only concern is SQS, then yes, EB supports SQS FIFO targets:

EventBridge lets you set a variety of targets—such as Amazon SQS standard and FIFO queues—which receive events in JSON format.

answered Feb 08 '22 at 23:37

Marcin

215,873
14
235
294

EB supports FIFO queues as a target, but I think the question was about FIFO queue deduplication? – zhe May 27 '23 at 06:55

score 0 · Answer 3 · answered Feb 09 '22 at 07:04

It is possible, but you have to configure the FIFO queue. It can detect duplicates based on the body of the message. See this docs.

FIFO queues help you avoid sending duplicates to a queue. If you retry the SendMessage action within the 5-minute deduplication interval, Amazon SQS doesn't introduce any duplicates into the queue.

The link goes on to state what configurations are required, so be sure to check it out.

Eventbridge - Use FIFO SQS for deduplication

3 Answers3