Asyncio aiohttp - parallel requests made per second

Question

I have checked this post (but I cannot reply): aiohttp: rate limiting parallel requests

First of all, I would like to say thanks @Sraw for his comments in the aforementioned post!

However, I have doubts if I am doing well with my current solution:

for convertedJson in allJsons:
   .....
   .....
   tasks.append(asyncio.ensure_future(fetch(env,typeRequest,convertedJson,headers,retries,execution,session)))
   await asyncio.sleep(1/800)

results = await asyncio.gather(*tasks,return_exceptions=True)
await session.close()

In my fetch function I am basically doing this:

async def fetch(env,typeRequest,request,headers,retries,execution,session): 
   .....
   .....
   async with session.post(url=VALIDATION_API_ENDPOINT,json=request["body_request"],headers=headers,timeout=None,allow_redirects=False) as response:
      ......
      ......
      jsonF=await response.json()
      return jsonF

The thing is that I want to make 800 requests per second. Am I doing it correctly?. I suppose that it is obvious that I am not able to know how long it is going to take to receive the responses. I'd like to know if I am already doing these 800 reqs/sec with this logic, because the server guys told me that I was actually doing around 400 reqs/sec (for me doesn't make sense).

Thanks you all! BR,

Answer 1

Asyncio does not gaurantee max execution time, only min execution time. This code:

for convertedJson in allJsons:
    ...
    tasks.append(asyncio.ensure_future(fetch(env,typeRequest,convertedJson,headers,retries,execution,session)))
    await asyncio.sleep(1/800)

Does the following:

• iterates convertedJson
• appends a futures object
• sleeps for at least 1/800 of a second

When you are delaying as little as 1.25 milli seconds, the time taken to perform all those other steps starts mattering. Your asyncio event loop has other stuff to service (like actually performing the requests) and it's starting to show: you are not actually getting through 800 requests in one second. You probably aren't even getting through adding them to the queue, since the steps in that loop will take a while. You definitely aren't getting through executing them all (as you saw).

Thus in summary:

• your other events aren't atomic, so even if you were only sleeping for 1.25ms you would start running into problems if they took too long (because you won't get through 800 iteratiions of your loop in 1s).
• asyncio's 1.25ms is best effort, but it's allowed to be longer. It probably is here.

What to do? 800 requests/second is clearly quite a struggle with your current setup. You might like to split generating those requests from the requesting code, and run an asyncio event loop in one thread dedicated to doing the fetching. You might like to try the semaphore solution, which might (guess) have a lower overhead. You might like to make more of your code awaitable (so the eventloop can better schedule your tasks). Lastly, you might like to batch the requests between a few threads, but that's a counsel of last resort, as the whole point about cooperative multitasking is to avoid multiple threads.

As a first bet, even before you start profiling your code to see what's taking so long, remove the sleep line and see how many requests/second you make. That will give you some idea of how much work is needed to get to the desired throughput.