1

I have a scenario in which my Spring boot microservice is scaled to 8 instances. Each service consume the message from MQ and makes a http call to third party service. However, the third party service has a rate limit i.e. it cannot accept more than 20 requests per second. Now that I have 8 instances of same service running its hard to keep track of count. Any solutions that could help me implement this in autoscale environment ?

Sourb
  • 11
  • 1
  • 1
    Does this solve your problem ? https://stackoverflow.com/questions/1407113/throttling-method-calls-to-m-requests-in-n-seconds – Shawrup Oct 26 '20 at 06:22

2 Answers2

0

I wouldn't advice to keep track of that state because that's virtually impossible. Have a look at Circuit Breakers which is included in Spring Cloud. They can add behaviour to outgoing calls including some retry and backoff settings, or return stubs if all else fails. Implementations include Spring-Retry and Netflix Hystrix among others. I guess it's still not 100% fault-tolerant but as you already use messaging that won't be an issue because if all retries fail you can nack the message.

There's also this introduction from Martin Fowler to Circuit Breakers which is really nice. Hope this might give you something new to concider.

Nico Van Belle
  • 4,911
  • 4
  • 32
  • 49
0

This blog post comes to my mind. They use a Token Bucket to control the flow. The use case sounds similar to yours.

Our connection to our SMS aggregator requires us to limit the rate that we send messages to their system.

jschnasse
  • 8,526
  • 6
  • 32
  • 72