How can i load proxies from a txt file in python?

Question

The proxies are the following format:

1.1.1.1:80
1.1.1.2:80

All of them are open on port 80.

Here is what i tried but without success it only gets 1 proxy from the file for some reasons.

import requests

s = requests.session()
proxy = set()
with open("proxies.txt", "r") as f:
    file_lines1 = f.readlines()
    for line1 in file_lines1:
        proxy.add(line1.strip())
        
proxies = {
    'http': 'http://'+line1
    }


r = requests.get('http://www.google.com/',proxies=proxies)

score 3 · Accepted Answer · answered Jul 27 '20 at 13:46

3

You should use the proxy variable that contains all the proxies. You can use the random module to pick random proxies


import requests, random

s = requests.session()
proxy = set()

with open("proxies.txt", "r") as f:
    file_lines1 = f.readlines()
    for line1 in file_lines1:
        proxy.add(line1.strip())
        
proxies = {
    'http': 'http://'+random.choice(list(proxy))
}


r = requests.get('http://www.google.com/',proxies=proxies)

answered Jul 27 '20 at 13:46

bigbounty

16,526
5
37
65

So when using multi threading and making requests it is gonna use 1 proxy/req? `def request() url='http://www.google.com/' req = request.get(url,proxies=proxies)` – Eduardo Decker Jul 27 '20 at 14:00
`proxy` variable can be shared between the threads. `random.choice` gives one element that is picked randomly – bigbounty Jul 27 '20 at 14:05
Do you have any ideeas how can i do that? to use proxies/threads? – Eduardo Decker Jul 27 '20 at 14:08
You can look at https://stackoverflow.com/questions/38280094/python-requests-with-multithreading – bigbounty Jul 27 '20 at 14:10
Thanks I was looking for this kind of solution – Farhan Ahmed Oct 09 '20 at 07:49

How can i load proxies from a txt file in python?

1 Answers1