1

I'm trying to access an API with aiohttp but something is causing this code to block each iteration.

def main():
    async with aiohttp.ClientSession() as session:
        for i, (image, target) in enumerate(dataset_val):
            image_bytes = pil_to_bytes(image)
            async with session.post('http://localhost:8080/predictions/resnet50', data=image_bytes) as resp:
                print(await resp.text())
                print(i, flush=True, end='\r')


asyncio.run(main())
Terv
  • 11
  • 1
  • 2
    Yes, `await` explicitly awaits the completion of the request before continuing the loop. To fire multiple requests in parallel and await their completion together, you need something like `asyncio.gather`. – deceze Aug 24 '22 at 08:50

1 Answers1

1

As explained by @deceze, await will wait for your result inside your loop. If you want to call everything at the same time, you need to call everything from an external loop and gather the results.

Here's a way of doing it

import asyncio
import aiohttp

async def call(session: aiohttp.ClientSession, url: str, image):
    image_bytes = pil_to_bytes(image)
    async with session.post(url, data=image_bytes) as response:
        return await response.text()


async def call_all(url:str, tasks: list):
    async with aiohttp.ClientSession() as session:
        results = await asyncio.gather(
            *[call(session, url, img) for img, target in tasks], 
            return_exceptions=True
        )
        return results


loop = asyncio.get_event_loop()
res = loop.run_until_complete(
    call_all('http://localhost:8080/predictions/resnet50', dataset_val)
)
RobinFrcd
  • 4,439
  • 4
  • 25
  • 49
  • Thanks @RobinFrcd, assuming tasks would be a too large dataset to fit in memory, what would be a good way to start processing the "results" parallel to while requests are processed? – Terv Aug 24 '22 at 13:25
  • You could use [Semaphores](https://stackoverflow.com/a/48486557/5692012) and tasks images could be replaced with images_path, and load the image only when needed. You could also take a look at [`Multiprocessing`](https://docs.python.org/3/library/multiprocessing.html) instead of asyncio here, it may fit your needs too. – RobinFrcd Aug 24 '22 at 13:33
  • @Terv Did it solve your problem ? – RobinFrcd Aug 26 '22 at 15:36
  • yes, thank you! I ended up also writing an async iterator that uses aiofiles to load files (otherwise iterating "tasks" will block the asyncio event loop). For processing the results concurrently I used an asyncio Queue with a producer-consumer scheme. – Terv Aug 31 '22 at 17:26