6

I am trying to scrape using proxy from proxymesh.com

I am currently using the following code, It opens Chrome and creates a javascript alert to input username and password. I am currently doing is manually everytime I run a new instance of the script.

If someone could please help in automating it.

There could be 2 ways to do it,

Either somehow pass the username and password through Chrome Options

OR

Somehow make webdriver switch to javascript alert and enter the username and password there.

Here is my code so far,

from selenium import webdriver
chrome_option = webdriver.ChromeOptions()
chrome_option.add_argument("--proxy-server=http://us.proxymesh.com:31280")

b = webdriver.Chrome('chromedriver.exe',
                          chrome_options=chrome_option)
"Do Something"

Thanks in advance

Md. Mohsin
  • 1,822
  • 3
  • 19
  • 34

1 Answers1

0

This is what has worked for me using Chrome and proxy authentication. I don't have a proxymesh account, so you'll have to verify if it works or not yourself:

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

proxy = {'address': 'us.proxymesh.com:31280',
         'usernmae': 'johnsmith123',
         'password': 'iliketurtles'}


capabilities = dict(DesiredCapabilities.CHROME)
capabilities['proxy'] = {'proxyType': 'MANUAL',
                         'httpProxy': proxy['address'],
                         'ftpProxy': proxy['address'],
                         'sslProxy': proxy['address'],
                         'noProxy': '',
                         'class': "org.openqa.selenium.Proxy",
                         'autodetect': False}

capabilities['proxy']['socksUsername'] = proxy['username']
capabilities['proxy']['socksPassword'] = proxy['password']

driver = webdriver.Chrome(executable_path=[path to your chromedriver], desired_capabilities=capabilities)
crookedleaf
  • 2,118
  • 4
  • 16
  • 38
  • @DaniilVolkov please see this post and it's comments for an explanation of what happened: https://stackoverflow.com/a/42633506/3126085 – crookedleaf Dec 27 '18 at 21:08
  • Yes, i saw,this method doesn`t work with chromedriver and proxy requiring authentication. I had to choose proxy server without authentication. – Daniil Volkov Dec 27 '18 at 21:42
  • 1
    @DaniilVolkov ah. i haven't tried anything new since these posts, which were made almost 2 years ago. i was hoping something would have changed by now, but maybe not. i ended up having to use proxies that allowed ip address authentication, myself. – crookedleaf Dec 28 '18 at 01:01