1

I make API calls with asyncio but I have a limit of 300 calls per minutes. How can I pause or limit asyncio? I tried with "semaphore" with no success. How can I do this?

from asyncio.tasks import sleep
from aiohttp import ClientSession
import asyncio
import time
import aiohttp

semaphore = asyncio.Semaphore(2)

async def get_sites(sites):
    tasks = [asyncio.create_task(fetch_site(s)) for s in sites] 
    return await asyncio.gather(*tasks)  

NCAVStock = []

async def fetch_site(url):
     async with ClientSession() as session:
         async with session.get(url) as resp:  
             data = await resp.json()
             data = data['symbol']
             NCAVStock.append(data)
             print(NCAVStock)
     return data

if __name__ == '__main__':
    List_Not_China = ['SRDX', 'AQB', 'CDTI', 'VRSN', 'MEC', 'NFG', 'KTOS', 'PRO', 'BEAT', 'HRB', 'JBLU', 'SRTS', 'PCRX', 'RVLV', 'CTSH', 'DHCNL', 'SYX', 'FARM', 'BAM', 'CALX', 'VTIQW', 'LKQ', 'ISR', 'GLDW', 'WORK', 'UTI', 'MXL', 'MTOR', 'CRWS', 'CHWY', 'GKOS', 'MDEX', 'AGI', 'LH', 'IDIV', 'CVEO', 'URI', 'FIX', 'RICK', 'ITW', 'STRT', 'SGLBW', 'EIX', 'AWX', 'ADSK', 'INS', 'MLHR', 'IIIV']
    sites = [
        f'http://financialmodelingprep.com/api/v3/financials/balance-sheet-statement/{company}?period=quarter&apikey=APIKEY'for company in List_Not_China
    ]
    data = asyncio.run(get_sites(sites))
mkrieger1
  • 19,194
  • 5
  • 54
  • 65
Félix Le Besnerais
  • 71
  • 4
  • Have you tried making 300 calls, then sleep till next minute? Or split into smaller intervals, like 10 seconds, depending on how fast api responds. – awesoon Dec 19 '21 at 18:24
  • Thanks for your answear. How can I increment this in python : "300 calls then sleep" ? I have a list about more than 2000 symbols. When you talk about "split". You mean split my list in 300 symbols for example for 2000 symbols split in 7 lists and do 7 times async with a sleep between them. – Félix Le Besnerais Dec 20 '21 at 14:03
  • Yes, you can try e.g. this one: https://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks and for every chunk just measure how much time did you spend and sleep till the next interval – awesoon Dec 20 '21 at 15:06
  • Does the `Limiter` from [this answer](https://stackoverflow.com/a/66521053/1600898) help? – user4815162342 Dec 21 '21 at 11:29

0 Answers0