I am trying to have an Akka stream where some of the stages are http calls. Since the http calls can (and will) fail at times, I would prefer to put relevant data to a separate queue (might even be external to my application), pull them after some time and consume them back in the stream.
I am using spray client for the http calls. But I am open to other clients (including the newer akka-http).
What is the cleanest way to achieve this? I can't do synchronous retries since that will block the stage. Currently I am putting the failed http requests to a raabitmq, adding a RMQ consumer source and a merge node before every http stage. This serves my purpose, but makes the stream code ugly and unmanageable. Even a simple linear topology becomes a complex graph with this approach.
I am using Akka stream (over a plain akka solution) primarily for easy composition and back pressure.
What alternatives do I have in this case (in and outside akka-stream). I would like to stick to an Akka solution for other design constraints.
