We are using Dapr PubSub in our microservices based application; We are intending to use Azure ServiceBus as the backing service in production.
When we attempt to handle a message, if one of the other services or the database is unavailable, then we throw an exception triggering a retry in Dapr. We want it to be robust so that if the dependency is only down briefly, the message still gets handled appropriately.
The problem is that it retries really quickly. Is there a way to slow down the rate of retries to be more like Dapr handles requests between microservices where it increases the gap between retries each time it fails. We don't want it using up too many resources doing rapid retries and it's not that time-critical; taking an extra few minutes would matter less than it not completing.
We have found a way to limit the number of retries so that it doesn't retry forever if a message is invalid but not a way to throttle the retries.
Is it possible to throttle the retries in this scenario?
We are using the Dapr API in .Net 5 / Asp.Net Core hosted on AKS in Linux containers.
