Python Selenium authentication proxy

Question

I'm trying to use a proxy with authentication on selenium. I have seen a lot of examples like this one

from selenium import webdriver
PROXY = "23.23.23.23:3128" # IP:PORT or HOST:PORT

options = webdriver.ChromeOptions()
options.add_argument('--proxy-server=%s' % PROXY)

chrome = webdriver.Chrome('./chromedriver',options=options) 
chrome.get("http://whatismyipaddress.com")

I'm not sure where I should enter the username and password or how, for the authentication. There is any documentation about that?

I have same problem, please check if this link could help: https://stackoverflow.com/questions/64829313/python-proxy-authentication-through-selenium-chromedriver — ah bon, Jan 20 '21 at 02:53
Does this answer your question? [Python proxy authentication through Selenium chromedriver](https://stackoverflow.com/questions/64829313/python-proxy-authentication-through-selenium-chromedriver) — GAP2002, Feb 09 '21 at 23:09

score 0 · Answer 1 · answered Feb 09 '21 at 23:09

As I have answered here.

If you need to use a proxy with no authentication (no username or password) with python and Selenium library with chromedriver you usually use the following code:

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % hostname + ":" + port)
driver = webdriver.Chrome(chrome_options=chrome_options)

It works fine unless proxy requires authentication. if the proxy requires you to log in with a username and password it will not work. In this case, you have to use more tricky solution that is explained below.

To set up proxy authentication we will generate a file and upload it to chromedriver dynamically using the following code below. This effectively create a chrome extension. This code configures selenium with chromedriver to use HTTP proxy that requires authentication with user/password pair.

import os
import zipfile

from selenium import webdriver


def create_chromedriver(PROXY_HOST, PROXY_PORT, PROXY_USER, PROXY_PASS, USER_AGENT):
    manifest_json = """
    {
    "version": "1.0.0",
    "manifest_version": 2,
    "name": "Chrome Proxy",
    "permissions": [
        "proxy",
        "tabs",
        "unlimitedStorage",
        "storage",
        "<all_urls>",
        "webRequest",
        "webRequestBlocking"
    ],
    "background": {
        "scripts": ["background.js"]
    },
    "minimum_chrome_version":"22.0.0"
    }
    """

    background_js = """
    var config = {
        mode: "fixed_servers",
        rules: {
        singleProxy: {
            scheme: "http",
            host: "%s",
            port: parseInt(%s)
        },
        bypassList: ["localhost"]
        }
    };

    chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});

    function callbackFn(details) {
        return {
            authCredentials: {
            username: "%s",
            password: "%s"
            }
        };
    }

    chrome.webRequest.onAuthRequired.addListener(
            callbackFn,
            {urls: ["<all_urls>"]},
            ['blocking']
    );
    """ % (PROXY_HOST, PROXY_PORT, PROXY_USER, PROXY_PASS)


    def get_chromedriver(use_proxy=True, user_agent=USER_AGENT):
        path = os.path.dirname(os.path.abspath(__file__))
        chrome_options = webdriver.ChromeOptions()
        if use_proxy:
            pluginfile = 'proxy_auth_plugin.zip'
            with zipfile.ZipFile(pluginfile, 'w') as zp:
                zp.writestr("manifest.json", manifest_json)
                zp.writestr("background.js", background_js)
            chrome_options.add_extension(pluginfile)
        if user_agent:
            chrome_options.add_argument('--user-agent=%s' % USER_AGENT)
        driver = webdriver.Chrome(
            os.path.join(path, 'chromedriver'),
            chrome_options=chrome_options)
        return driver

    driver = get_chromedriver(use_proxy=True)
    # driver.get('https://www.google.com/search?q=my+ip+address')
    driver.get('https://httpbin.org/ip')


user_agent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36'
create_chromedriver('192.168.3.2', 8080, 'user', 'pass', user_agent)

Function get_chromedriver returns configured selenium web driver that you can use in your application. This code is tested and works just fine.

To implement this into your own code just call the function create_chromedriver with args of host, port, user, pass and user-agent. If you do not want to use proxy or user-agent just edit get_chromedriver accordingly so the args are false.

Python Selenium authentication proxy

1 Answers1