How to change proxy on my webdriver multiple times on a single session?

Question

I am working on a bot. I want the bot to change the proxy of the webdriver every 50 searches. I have an API that requests the proxy and the socket, i store those variables and so far i have been using firefox profiles to set it up but that doesnt work very well.

So given the fact that i already have a viable source of proxies and ports, can you tell me any way i can change the proxy without crashing the webdriver and doing it on a single session?

Previous attempts:

I tried setting up a firefox profile this way:

regions = {
    'US': '', #USA is the default server
    'Australia': #json response through the api,
    'Canada': #json response through the api,
    'France': #json response through the api,
    'Germany': #json response through the api,
    'UK': #json request response the api
}    
for region in regions:
        fp = webdriver.FirefoxProfile()
        if(regions[region] != ''):
            fp.set_preference("network.proxy.type", 1)
            fp.set_preference("network.proxy.socks", regions[region])
            fp.set_preference("network.proxy.socks_port", port)

This caused me some problems and i had to start a new session each time i wanted to swap a proxy. So i tried to change the proxy through the Firefox options (options - general - connection settings) but turns out the popup that appears on the screen after clicking the connection settings button is not accessible through selenium or javascript (xul file).

@DebanjanB I have re formatted the question and hopefully this clears up things and what i am trying to do rather then just saying what i have been doing and what has not worked for me. — SuperSimplePimpleDimple, Feb 10 '19 at 14:37
I would suggest to put back the thesis within the question as that shares that research, code attempts, and results. This saves us from reiterating obvious answers, and most of all it helps you get a more specific and relevant answer! — undetected Selenium, Feb 10 '19 at 14:57
@DebanjanB I am making a bot and when I need to log in with many accounts many times. If I drop the session each time I want to change the proxy I have to log in again. Too many log ins in a short period of time causes suspicion by the algorithm causing them to believe that someone is using someone else's account (which is true since I bought these accounts) and it locks them. — SuperSimplePimpleDimple, Feb 10 '19 at 18:49
@DebanjanB I think i have found a way to do it through an extension. Can you show me the code on Python for starting the session with an extension? — SuperSimplePimpleDimple, Feb 12 '19 at 21:13
I think that here is a solutions of the same problem If it is not sorry about it : https://stackoverflow.com/questions/29776607/python-selenium-webdriver-changing-proxy-settings-on-the-fly?noredirect=1&lq=1 @SuperSimplePimpleDimple — sayhan, Feb 13 '19 at 08:56
@SelcukAyhan Seems your pointer worked for OP and you may want to add a canonical answer. — undetected Selenium, Feb 13 '19 at 19:34
@DebanjanB I think it is now not necessary because he answer it already. :( — sayhan, Feb 14 '19 at 10:05
@SelcukAyhan It's not only about answering to a question. StackOverflow is an effort to build up a library of good question and canonical answers to those questions. — undetected Selenium, Feb 14 '19 at 10:07

score 5 · Answer 1 · edited Feb 16 '19 at 16:12

I was able to solve this issue by setting up the preferences through JS on aboutLconfig and then used execute_script in selenium to deploy the js through python:

regions = {
'US': '', #USA is the default server
'Australia': #json response through the api,
'Canada': #json response through the api,
'France': #json response through the api,
'Germany': #json response through the api,
'UK': #json request response the api
}   
    for region in regions:
        driver.get("about:config")
        time.sleep(3)
        driver.find_element_by_css_selector("window#config deck#configDeck vbox#warningScreen vbox#warningBox.container vbox.description hbox.button-container button#warningButton.primary").click()
        time.sleep(3)
        driver.execute_script('var prefs = Components.classes["@mozilla.org/preferences-service;1"].getService(Components.interfaces.nsIPrefBranch); prefs.setIntPref("network.proxy.type", 1); prefs.setCharPref("network.proxy.socks", "' + regions[region] + '"); prefs.setIntPref("network.proxy.socks_port", 9998);')
        time.sleep(3)
        driver.get('https://www.whatsmyip.com/')
        time.sleep(10)

With the script im executing i am changing service value of the socks host and socks host with the region and the port respectively.

It essentially is the same as setting up a profile through selenium but this way you do it while the bot is running. You could also change user agent and pretty much anything you'd like this way.

Can you please provide additional context regarding how it solves the problem which would improve the answer's long-term value and benefit the future readers. — undetected Selenium, Feb 13 '19 at 19:33
@DebanjanB One downside i found here was the fact that the script must not be split down into more then 1 line (in python at least) cause it will spit out an "expecting ) after args[]" error. — SuperSimplePimpleDimple, Feb 13 '19 at 19:39
@SuperSimplePimpleDimple, you could avoid the `about:config` page by switching to the browser context before running the script: https://stackoverflow.com/questions/29776607/python-selenium-webdriver-changing-proxy-settings-on-the-fly/54713821#54713821 — Florent B., Feb 15 '19 at 17:02

score 5 · Accepted Answer · answered Feb 14 '19 at 10:07

According to this topic, here is your solution :

Solution link : Python Selenium Webdriver - Changing proxy settings on the fly

var setupScript=`var prefs = Components.classes["@mozilla.org/preferences-service;1"]
.getService(Components.interfaces.nsIPrefBranch);

prefs.setIntPref("network.proxy.type", 1);
prefs.setCharPref("network.proxy.http", "${proxyUsed.host}");
prefs.setIntPref("network.proxy.http_port", "${proxyUsed.port}");
prefs.setCharPref("network.proxy.ssl", "${proxyUsed.host}");
prefs.setIntPref("network.proxy.ssl_port", "${proxyUsed.port}");
prefs.setCharPref("network.proxy.ftp", "${proxyUsed.host}");
prefs.setIntPref("network.proxy.ftp_port", "${proxyUsed.port}");
                  `;    

//running script below  
driver.executeScript(setupScript);

//sleep for 1 sec
driver.sleep(1000);

How to change proxy on my webdriver multiple times on a single session?

2 Answers2

Linked