0

I am trying to scrape ali express. And as it is known, my ip could be banned if a scraping attempt is detetced... That's why I am trying to hide my IP with a proxy in selenium wire , but when I check in a IP website, it gives me the same IP as when I am not using a proxy. This is my source code :

from seleniumwire import webdriver
import requests


def gimme_proxy():
    orilist = ["104.28.18.46:80","104.31.6.146:80","104.28.4.43:80","136.228.165.138:8080"]
    finalist = []
    for occurence in orilist:
        # finalist.append({"http": occurence, "https": occurence})
        finalist.append({ "https": occurence})
    for proxy in finalist:
        if proxy_try(proxy) == 0:
            continue
        else:
            print("Mission passed !!!!!!!!!!!!!!!!!!!")
            print(proxy)
            return proxy



def get_page(url,proxy,useragent):
    options = {"proxy": proxy}
    driver = webdriver.Chrome(seleniumwire_options=options)
    driver.header_overrides = {"User-Agent": useragent}
    driver.get(url)
    return driver.page_source


def proxy_try(proxy):
    try:
        requests.get("http://google.com",proxies=proxy)
    except Exception :
        print("iddek fiih")
        return 0
    else:
        print("lqadyya msllka waqila")
    return 1

useragent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36'
get_page("https://whatismyipaddress.com/fr/mon-ip",gimme_proxy(),useragent)

thanks in advance

Anas Saada
  • 1
  • 1
  • many free proxies don't hide IP. They can sent original IP in extra header and server may get it. – furas Jan 04 '21 at 17:17
  • Thanks for the reply Where do you think I can get a working proxy that can hide my Ip ? – Anas Saada Jan 04 '21 at 18:58
  • first: I tried your code and it doesn't work for me. I had to use `proxy = {"https": 'https://' + address, "http": 'http://' + address}` to start working. And I'm not sure if you check it correctly - you create proxy for `https` but you test it with google using `http` so it may not test proxy. – furas Jan 04 '21 at 19:19
  • As for proxy server - for me works [TOR network](https://www.torproject.org/) but I don't if you would havet to install `TOR browser` to get `TOR proxy` - I use Linux Mint and it has `proxy` as separated elements. And there is also Python module to change IP in TOR proxy [Make requests using Python over Tor](https://stackoverflow.com/questions/30286293/make-requests-using-python-over-tor) – furas Jan 04 '21 at 19:23
  • there are also portals which offers access to proxies for money (using API) but sometimes they give few requests every month for free ie. 1000 requests per month in [scraperapi](https://www.scraperapi.com/?fp_ref=dhxwo) – furas Jan 04 '21 at 19:43

0 Answers0