How to send HTTP requests without getting the status/source code - Python 3

Question

I know when you want to send HTTP requests in Python 3 , you use the following code:

import requests
requests.get(url = "http://example.com")

and the program waits for the website to send back it's status / source code. But I'm trying to write a program that sends HTTP requests but doesn't wait for the website to reply.How can I do that?

EDIT I tried to use TCP and UDP but neither works since I get the status of the website after the packets are being sent.

Have a look at https://github.com/kennethreitz/grequests and https://github.com/requests/requests-threads — Sven Rusch, Jun 19 '18 at 10:39
@SvenFestersen I did have a look at them but they both wait for the response because they print the status of the website.I don't even want to see whether the website is down or not. — Borna Ghahnoosh, Jun 19 '18 at 10:48

score 1 · Answer 1 · answered Jun 19 '18 at 10:49

1

If you dont care about the response at all, you can use multiprocessing pool for this.

import multiprocessing
import requests

p = multiprocessing.Pool(processes = 1)
p.apply_async(requests.get, ['https://google.com'])

I assume the only purpose you would want to do this is when you want to make asynchronous calls without wating for a call to finish to start another so you can increase the number of processes and call asynchronously in a for loop as:

import multiprocessing
import requests

p = multiprocessing.Pool(processes = 4)
for i in range(10):
    p.apply_async(requests.get, ['https://google.com'])

Just an example, number of processes and async call can vary based on your processor capability.

answered Jun 19 '18 at 10:49

MohitC

4,541
2
34
55

Thank you this helped me a lot.But I'm trying to make something like a server (not exactly a server) and this code doesn't seem to work if a send requests more than 1000 times.Is there a way to handle that? – Borna Ghahnoosh Jun 19 '18 at 13:04
What exactly doesnt work when you send request more than 1000 times? – MohitC Jun 19 '18 at 13:04
It gives the error:"Too many pages open" – Borna Ghahnoosh Jun 19 '18 at 13:15
If this error is given by your server while serving the request then this is not related to the question you asked. For the question you asked, this is the solution – MohitC Jun 19 '18 at 13:16
Not by the server by the python program – Borna Ghahnoosh Jun 19 '18 at 13:23
That is unliekely. Can you post the entire stack trace? – MohitC Jun 19 '18 at 13:25

How to send HTTP requests without getting the status/source code - Python 3

1 Answers1