-1

I have a list of about 70 items and I need to make an api request for each item. I am currently using a for loop to go through each item in that list and make the api request. Is there a more efficient way to do this? I would imagine there is a way to make the api requests simultaneously instead of looping.

JackW24
  • 135
  • 9
  • 3
    You need some sort of loop. You might use threads to make the requests concurrently instead of serially, but depending on how long each request takes, the overhead may not be worth it. – chepner Oct 14 '22 at 19:17
  • 1
    You could write the code using python's async services (that would include the api request itself) or use a `mulitprocessing.pool.ThreadPool`. – tdelaney Oct 14 '22 at 19:20
  • @chepner You can make many more async requests than you can with threading – Tom McLean Oct 14 '22 at 19:23
  • Does this answer your question? [Fastest parallel requests in Python](https://stackoverflow.com/questions/57126286/fastest-parallel-requests-in-python) – Gino Mempin Oct 15 '22 at 03:40

3 Answers3

0

You should use some kind of parallelism if possible, such as the Pool class.

EJoshuaS - Stand with Ukraine
  • 11,977
  • 56
  • 49
  • 78
0

I would use httpx

async def send_request(item):
    async with httpx.AsyncClient() as client:
       r = await client.get('https://www.example.com/', data=item)
    return r.json()

async def main():
    data = [{"hi": 1}, {"hi": 2}]
    coros = [send_request(x) for x in data]
    results = await asyncio.gather(*coros)
    return results


loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Tom McLean
  • 5,583
  • 1
  • 11
  • 36
0

you can use standard python library threading:

def name_func(param):
    do something

thread1=threading.Thread(target=name_func, args =(a,))
thread2=threading.Thread(target=name_func, args =(b,))

thread1.start()
thread2.start()
doppelherz7
  • 11
  • 1
  • 3