How to do parallel requests using python for 1000 requests?

Asked Jul 08 '19 at 08:26

Active Jul 08 '19 at 13:06

Viewed 465 times

How to do parallel requests using python?

My code so far:

x = ("address")
x.signin(signin.request())
# this code signs in the user and generate the response how to do a parallel response.

Looked into those but couldn't figure out where to use that

import asyncio
import requests

 async def main():
     loop = asyncio.get_event_loop()
     futures = [
        loop.run_in_executor(
            None, 
            requests.get, 
            'http://example.org/'
        )
        for i in range(20)
    ]
    for response in await asyncio.gather(*futures):
        pass

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

looked into this thread also:
What is the fastest way to send 100,000 HTTP requests in Python?

edited Jul 08 '19 at 13:06

Ivan Vinogradov

4,269
6
29
39

asked Jul 08 '19 at 08:26

is it on linux or windows? – MEdwin Jul 08 '19 at 08:44
windows, but its same in both – Jul 08 '19 at 08:53
i have been successfuling using the `grequests` module in windows. Just `map` the request into a list. – MEdwin Jul 08 '19 at 09:17
can you post some link of ex of it – Jul 08 '19 at 09:54
here you go: https://stackoverflow.com/questions/32829798/python-how-can-i-send-multiple-http-requests-and-receive-the-response. Let me know if it works. – MEdwin Jul 08 '19 at 09:59

How to do parallel requests using python for 1000 requests?

0 Answers0