0

I have two services, one is the producer (Service A) and one is a consumer (Service B). So Service A will produce a message which will be published to Amazon SQS service and then it will be delivered to Service B as it has subscribed to the queue. So, this works fine until I have a single instance of Service B.

But when I start another instance of Service B, such that now there are 2 instances of Service B, both of which are subscribing to the same queue, as it is the same service, I observe that the messages from SQS are now being delivered in round-robin fashion. Such that at a given time, only one instance of Service B receives the message that is published by Service A. I want that when a message is published to this queue, it should be received by all the instances of Service B.

How can we do this? I have developed these services as Springboot applications, along with Spring cloud dependencies.

Please see the diagram below for reference. enter image description here

Loui
  • 533
  • 17
  • 33
  • 1
    This is not possible. Amazon SQS only enables one consumer to read a message. Something like different consumer groups in Kafka is not available in SQS. If you want something like this, you should use SNS together with several SQS instances. – jAC Jul 13 '21 at 14:25
  • Does this answer your question? [Using Amazon SQS with multiple consumers](https://stackoverflow.com/questions/30296587/using-amazon-sqs-with-multiple-consumers) – jAC Jul 13 '21 at 14:31
  • ya, I read that Amazon SQS only enables one consumer to read a message, but how does messaging work with services that have multiple instances running? – Loui Jul 13 '21 at 14:33
  • 1
    If you have multiple instances of one application, it is ensured that at least one of those instances receives and handles the message but it is not guaranteed which instance it is. The question is, why do you need this? It seems to be that you're fixing an issue which is not really the fault of SQS. – jAC Jul 13 '21 at 14:57
  • no i am not fixing any issue, due to present limitation we have to get the message delivered to all the running instances. Therefore, i wanted to see the possibilities. – Loui Jul 13 '21 at 17:51
  • You appear to be confusing Amazon SNS (publish-subscribe) with Amazon SQS (push into queue, pull from queue). – John Rotenstein Jul 13 '21 at 22:59
  • @Loui So you're fixing another issue. The issue is that your using a scalable/microservice(?) architecture without considering things like statelessness. You should think about using a centralized cache or something similar to avoid such issues. This is not an issue about SNS/SQS but about your system design so you should try to fix that instead. – jAC Jul 14 '21 at 06:14

3 Answers3

2

While your message may appear to be read in a round robbin fashion, they are not actually consumed in a round robin. SQS works by making all messages available to any consumer (that has the appropriate IAM permissions) and hides the message as soon as one consumer fetches the message for a pre-configured amount of time that you can configure, effectively "locking" that message. The fact that all of your consumer seem to be operating in a round robin way is most likely coincidental.

As others have mentioned you could use SNS instead of SQS to fanout messages to multiple consumers at once, but that's not as simple a setup as it may sound. If your service B is load balanced, the HTTP endpoint subscriber will point to the Load Balancer's DNS name, and thus only one instance will get the message. Assuming your instances have a public IP, you could modify your app so that it self-registers as an HTTP subscriber to the topic when the application wakes up. The downsides here are that you're not only bypassing your Load Balancer, you're also losing the durability guarantees that come with SQS since an SNS topic will try to send the message X times, but will simply drop the message after that.

An alternative solution would be to change the message hiding timeout setting on the SQS queue to 0, that way the message is never locked and every consumer will be able to read it. That will also mean you'll need to modify your application to a) not process messages twice, as the same message will likely be read more than once by the time it has finished processing and b) handle failure gracefully when one of the instance deletes the message from the queue and other instances try to delete that message from the queue after that.

Alternatively, you might want to use some sort of service mesh, or service discovery mechanism so that instances can communicate between each other in a peer-to-peer fashion so that one instance can pull the message from the SQS queue and propagate it to the other instances of the service.

You could also use a distributed store like Redis or DynamoDB to persist the messages and their current status so that every instance can read them, but only one instance will ever insert a new row.

Ultimately there's a few solutions out there for this, but without understanding the use-case properly it's hard to make a hard recommendation.

William Perron
  • 1,288
  • 13
  • 18
  • If you go the route of a single SQS queue and set the queue's visibility timeout to zero, which instance of service B deletes the message from the SQS queue? You're going to need some additional coordination. Also, each message will remain until the very last consumer (however you determine that) deletes it, which couples the consumers. – jarmod Jul 13 '21 at 15:09
  • Absolutely, it's definitely not what I would choose to use, but it is a possibility – William Perron Jul 13 '21 at 16:01
  • @WilliamPerron this is really a great answer, thanks for providing various possibilities, helps me to think more. what exactly do you want to know more about the use-case? does my comment from the above section provides your more insights? – Loui Jul 13 '21 at 18:17
  • Well I'd need to know it's an issue that a single instance process each message. Is it session data? is it something that needs to be read immediately, or processed asynchronously? is it kept in memory, or written to a local disk? When you get into these architecture decisions, the answer is always "it depends" and without knowing all the constraints it's hard to make a concrete recomendation. – William Perron Jul 13 '21 at 19:05
1

Implement message fanout using Amazon Simple Notification Service (SNS) and Amazon Simple Queue Service (SQS). There is a hands-on Getting Started example of this.

Here's how it works: in the fanout model, service A publishes a message to an SNS topic. Each instance of service B has an associated SQS queue which is subscribed to that SNS topic. The published message is delivered to each subscribed queue and hence to each instance of service B.

jarmod
  • 71,565
  • 16
  • 115
  • 122
  • This doesn't address the question; either the target of the SNS topic is an SQS queue, which brings OP back exactly to where they are, or it's an HTTP endpoint, which does nothing to solve the question "how can all instances of the same service receive the message" – William Perron Jul 13 '21 at 14:44
  • @WilliamPerron as I understand it, the OP wants a single message from service A to be received by all instances of service B. In the fanout model, service A publishes a message to an SNS topic. Each instance of service B has an associated SQS queue which is subscribed to that SNS topic. The published message is delivered to each subscribed queue and hence to each instance of service B. – jarmod Jul 13 '21 at 14:56
  • That is _if_ every single instance is registered _independently_ in the SNS topic, which is a very important nuance to bring as it impacts either config management or more likely user code if the number of hosts is dynamic (as in the case of an autoscaler for example) – William Perron Jul 13 '21 at 15:00
  • 1
    having a dedicated SQS queue for _each_ instance of Service B will also become quite expensive pretty quickly, I wouldn't recommend this approach. Ultimately, I'm more interested in hearing the use-case for this, typically instances of the same service should be stateless and be able to process messages independently. Having each instance process all messages suggest to me there's some hidden context here that isn't provided by OP _or_ OP is going down the wrong direction with their architecture – William Perron Jul 13 '21 at 15:02
  • Thanks @WilliamPerron and jarmod for your inputs. And Willam you rightly guessed the major reason for doing this is due to limitation of service instances not being stateless. There is some state that will be maintained in and therefore, when the messages are delivered to all the running instances, the instances for which this message won't be relevant will not consume it. But as i said, this is majorly due to states being maintained and we cannot do much at moment to make the service B as stateless. In future, the goal is to refactor the service and make it stateless. – Loui Jul 13 '21 at 17:58
1

If you are interested in building functionality like this, use SNS, not SQS. We have a Spring BOOT example that shows how to build a web app that lets users sign up for email subscriptions and then when a message is published, all subscribed emails get the message.

The purpose of this example is to get you up and running building a Spring BOOT app using the Amazon Simple Notification Service. That is, you can build this app with Spring BOOT and the official AWS Java V2 API:

enter image description here

Creating a Publish/Subscription Spring Boot Application

smac2020
  • 9,637
  • 4
  • 24
  • 38