Python Selenium Webdriver - Changing proxy settings on the fly

Question

I'm currently successfully using the code below to use a proxy with the Selenium webdriver. Unfortunately, I can't seem to make it change the proxy settings without restarting the whole browser. I had hoped that simply updating the proxy settings, just like I did to set the proxy to start with, would change the proxy, but it doesn't seem to work. Any help on this subject would be greatly appreciated.

profile = webdriver.FirefoxProfile()
profile.set_preference("network.proxy.type", 1)
profile.set_preference("network.proxy.http", proxyAddress)
profile.set_preference("network.proxy.http_port", proxyPort)
profile.update_preferences()
driver = webdriver.Firefox(firefox_profile=profile)

create a local proxy which chains to your proxies. When necessary ask your local proxy to change the "exit". — user37203, Apr 21 '15 at 15:49
That seems overly complicated. I hope there's an easier way of doing this. — tobloef, Apr 21 '15 at 16:42
it actually sounds like 10 lines of code, there's a python library for anything out there.. — user37203, Apr 21 '15 at 17:32
Did you by any chance find a solution for this? Or a python lib that solves the problem as @user37203 describes? Just asking, before implementing it on my own... — stylesuxx, Oct 07 '16 at 00:59
@stylesuxx Nope. I never found a library that did this, and I dropped the project, so I never got around to solving the problem myself. — tobloef, Oct 07 '16 at 01:02
Thanks for your quick response, I'll look into alternatives, maybe proxychains has some way of inter process communication... — stylesuxx, Oct 07 '16 at 01:12

Bob Kimani · Accepted Answer · 2018-02-15T23:12:24.703

12

This is a slightly old question. But it is actually possible to change the proxies dynamically thru a "hacky way" I am going to use Selenium JS with Firefox but you can follow thru in the language you want.

Step 1: Visiting "about:config"

driver.get("about:config");

Step 2 : Run script that changes proxy

var setupScript=`var prefs = Components.classes["@mozilla.org/preferences-service;1"]
.getService(Components.interfaces.nsIPrefBranch);

prefs.setIntPref("network.proxy.type", 1);
prefs.setCharPref("network.proxy.http", "${proxyUsed.host}");
prefs.setIntPref("network.proxy.http_port", "${proxyUsed.port}");
prefs.setCharPref("network.proxy.ssl", "${proxyUsed.host}");
prefs.setIntPref("network.proxy.ssl_port", "${proxyUsed.port}");
prefs.setCharPref("network.proxy.ftp", "${proxyUsed.host}");
prefs.setIntPref("network.proxy.ftp_port", "${proxyUsed.port}");
                  `;    

//running script below  
driver.executeScript(setupScript);

//sleep for 1 sec
driver.sleep(1000);

Where use ${abcd} is where you put your variables, in the above example I am using ES6 which handles concatenation as shown, you can use other concatenation methods of your choice , depending on your language.(The SetupScript is a string containing the script to be runned enclosed by ``)

Step 3: : Visit your site

driver.get("https://whatismyip.com");

Explanation:the above code takes advantage of Firefox's API to change the preferences using JavaScript code.

edited Feb 15 '18 at 23:12

answered Feb 15 '18 at 21:28

Bob Kimani

1,114
1
20
38

1

I admittedly haven't tested the solution, and the project was abandoned long ago, but I'll accept the answer anyway. Thanks! – tobloef Feb 16 '18 at 21:58
2

I have tested this solution and it works great as such. – Tarun Lalwani May 24 '18 at 15:59
1

Can you provide a python version? Thank – erotavlas Mar 17 '19 at 22:15
Can you provide a chrome version?? please. – wataru Dec 11 '20 at 02:04
Sorry for a stupid question but where do you run this script? And how do you run it? Can you use variables in it or do you need to create one script for each proxy? I want to use it with Java, if that makes a difference. – d-b Dec 12 '20 at 23:53
@wataru sadly id don't know how to do it in chrome. – Bob Kimani Dec 14 '20 at 08:24
@d-b You can use java code to nagivate to about:config then use driver.execute script to run the code above. You can use a function that returns a string of the script for each proxy. – Bob Kimani Dec 14 '20 at 08:26

score 5 · Answer 2 · answered Feb 15 '19 at 16:56

To set a proxy on the fly with Firefox:

def set_proxy(driver, http_addr='', http_port=0, ssl_addr='', ssl_port=0, socks_addr='', socks_port=0):

    driver.execute("SET_CONTEXT", {"context": "chrome"})

    try:
        driver.execute_script("""
          Services.prefs.setIntPref('network.proxy.type', 1);
          Services.prefs.setCharPref("network.proxy.http", arguments[0]);
          Services.prefs.setIntPref("network.proxy.http_port", arguments[1]);
          Services.prefs.setCharPref("network.proxy.ssl", arguments[2]);
          Services.prefs.setIntPref("network.proxy.ssl_port", arguments[3]);
          Services.prefs.setCharPref('network.proxy.socks', arguments[4]);
          Services.prefs.setIntPref('network.proxy.socks_port', arguments[5]);
          """, http_addr, http_port, ssl_addr, ssl_port, socks_addr, socks_port)

    finally:
        driver.execute("SET_CONTEXT", {"context": "content"})

Usage:

 driver = webdriver.Firefox()

 set_proxy(driver, http_addr="212.35.56.21", http_port=8080)

 driver.get("http://....")

 set_proxy(driver, http_addr="212.35.56.22", http_port=8888)

 driver.get("http://....")

I'm trying to use your solution but i keep getting `KeyError: 'SET_CONTEXT'` and i can't figure what i'm doing wrong. Can't even find the documentation on these functions. Any help is much appreciated. Thanks! — Mike Vlad, Apr 18 '19 at 10:19
A simpler method for getting into chrome context temporarily is `with driver.context(driver.CONTEXT_CHROME): <...>` — ivan_pozdeev, Apr 02 '20 at 19:34

Samuel Chen · Answer 3 · 2021-02-01T01:36:48.107

To change proxy on the fly with Chrome (work on selenium 3.141.0, key point is driver.start_session(cap)):

   proxy = get_new_proxy()     # x.x.x.x:y
   
   c = {
       "proxyType": "MANUAL",
       "httpProxy": proxy,
       "sslProxy": proxy
   }
   
   cap = webdriver.DesiredCapabilities.CHROME.copy()
   cap['proxy'] = c
   driver.start_session(cap)
   try:
       b.get('https://whatismyip.com')
   except Exception as e:
       print(e)

p.s. selenium.webdriver.common.proxy.Proxy.add_to_capabilities() may also be used when specifying proxy (so you do not need to use the c dict above.)

Python Selenium Webdriver - Changing proxy settings on the fly

3 Answers3

Linked