3

I have the following method that generates data for me:

async def fetch_url(self, video_id):
    data = await self.s3.generate_presigned_url(...video_id...)
    return data

def convert_to_json(self, data):
    loop = asyncio.get_event_loop()
    tasks = []
    urls = [row[0] for row in data]
    for url in urls:
        tasks.append(fetch_url(url))
    loop.run_until_complete(asyncio.gather(*tasks))
    loop.close()

How to store result from fetch_url in some list?

1 Answers1

3

asyncio.gather:

… If all the tasks are done successfully, the returned future’s result is the list of results (in the order of the original sequence, not necessarily the order of results arrival). …

That is, if you await the gathered result, you will get a list of fetched data


run_until_complete:

Return the Future’s result, or raise its exception.

That is run_until_complete will return the result gather, which is the list of fetched data.


The stored result is simply:

...
all_data = loop.run_until_complete(asyncio.gather(*tasks))
loop.close()
return all_data
kennytm
  • 510,854
  • 105
  • 1,084
  • 1,005
  • Thank you very much! After I run this code I got `TypeError: object str can't be used in 'await' expression` inside fetch_url. How can I fix that? –  Apr 05 '17 at 09:53
  • @ipetr This means `fetch` is not really an async method. You need to use an async request library like [aiohttp](http://aiohttp.readthedocs.io/en/stable/client.html#aiohttp-client) or http://stackoverflow.com/questions/22190403/how-could-i-use-requests-in-asyncio. – kennytm Apr 05 '17 at 10:09
  • I use aiohttp. `fetch` is return `await self.s3.generate_presigned_url(...video_id...)`. Update my question –  Apr 05 '17 at 10:40
  • @ipetr well `generate_presigned_url` is not async either. – kennytm Apr 05 '17 at 12:35