Using Selenium RemoteWebDriver behind corporate proxy

Question

How can I connect to a selenium grid such as BrowserStack via RemoteWebDriver from behind a corporate proxy?

The application under test is outside the proxy and freely accessible from BrowserStack.

This Using Selenium RemoteWebDriver behind corporate proxy (Java) stackoverflow question asked the same question but I couldn't follow the accepted answer.

score 12 · Accepted Answer · answered Jan 20 '16 at 19:35

I managed to get something working based on the accepted answer in the linked stackoverflow question, here's my implementation in case anyone else is stuck on the same problem:

Example

import java.net.InetAddress;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.UnknownHostException;
import java.util.HashMap;
import java.util.Map;

import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.NTCredentials;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.HttpClientBuilder;
import org.openqa.selenium.remote.CommandInfo;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.HttpCommandExecutor;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.remote.http.HttpClient.Factory;

public class Example {
    public RemoteWebDriver connectViaProxy(DesiredCapabilities caps) {
        String proxyHost = "?";
        int proxyPort = 8080;
        String proxyUserDomain = "?";
        String proxyUser = "?";
        String proxyPassword = "?";

        URL url;

        try {
            url = new URL("http://bsuser:bspassword@hub.browserstack.com/wd/hub");
        } catch (MalformedURLException e) {
            throw new RuntimeException(e.getMessage(), e);
        }

        HttpClientBuilder builder = HttpClientBuilder.create();

        HttpHost proxy = new HttpHost(proxyHost, proxyPort);

        CredentialsProvider credsProvider = new BasicCredentialsProvider();

        credsProvider.setCredentials(new AuthScope(proxyHost, proxyPort), new NTCredentials(proxyUser, proxyPassword, getWorkstation(), proxyUserDomain));

        if (url.getUserInfo() != null && !url.getUserInfo().isEmpty()) {
            credsProvider.setCredentials(new AuthScope(url.getHost(), (url.getPort() > 0 ? url.getPort() : url.getDefaultPort())), new UsernamePasswordCredentials(url.getUserInfo()));
        }

        builder.setProxy(proxy);
        builder.setDefaultCredentialsProvider(credsProvider);

        Factory factory = new MyHttpClientFactory(builder);

        HttpCommandExecutor executor = new HttpCommandExecutor(new HashMap<String, CommandInfo>(), url, factory);

        return new RemoteWebDriver(executor, caps);
    }

    private String getWorkstation() {
        Map<String, String> env = System.getenv();

        if (env.containsKey("COMPUTERNAME")) {
            // Windows
            return env.get("COMPUTERNAME");         
        } else if (env.containsKey("HOSTNAME")) {
            // Unix/Linux/MacOS
            return env.get("HOSTNAME");
        } else {
            // From DNS
            try
            {
                return InetAddress.getLocalHost().getHostName();
            }
            catch (UnknownHostException ex)
            {
                return "Unknown";
            }
        }
    }
}

MyHttpClientFactory

import java.net.URL;

import org.apache.http.impl.client.HttpClientBuilder;
import org.openqa.selenium.remote.internal.ApacheHttpClient;

public class MyHttpClientFactory implements org.openqa.selenium.remote.http.HttpClient.Factory {
    final HttpClientBuilder builder; 

    public MyHttpClientFactory(HttpClientBuilder builder) {
        this.builder = builder;
    }

    @Override
    public org.openqa.selenium.remote.http.HttpClient createClient(URL url) {
        return new ApacheHttpClient(builder.build(), url);
    }
}

This was extremely useful, thanks. I haven't found any other documentation talking about this issue. It worked well in our corporate network. — Tao Starbow, Mar 04 '17 at 05:20
The Apache Http Client is removed in selenium any alternative ways to set the remote driver proxy — krishnakumar, Aug 30 '19 at 10:01
I've been wondering myself, haven't had a chance to look into this yet. — Andrew Sumner, Sep 02 '19 at 09:08

score 0 · Answer 2 · answered Apr 13 '18 at 06:44

0

Adding to the answer above by Andrew, to make this work with Appium change the

HttpCommandExecutor executor = new HttpCommandExecutor(new HashMap<String, CommandInfo>(), url, factory);

to

HttpCommandExecutor executor = new HttpCommandExecutor(MobileCommand.commandRepository, url, factory);

answered Apr 13 '18 at 06:44

Sravan Kumar

1

The question doesn't ask for Appium. – Pirate X Apr 13 '18 at 07:12

score 0 · Answer 3 · answered May 15 '18 at 15:41

I've reworked Andrew Sumner's solution slightly and taken some out in case someone like me wants to just quickly funnel their WebDriver traffic through Fiddler to see the traffic.

import java.net.MalformedURLException;
import java.net.URL;
import java.util.HashMap;

import org.apache.http.HttpHost;
import org.apache.http.impl.client.HttpClientBuilder;
import org.openqa.selenium.ie.InternetExplorerOptions;
import org.openqa.selenium.remote.CommandInfo;
import org.openqa.selenium.remote.HttpCommandExecutor;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.remote.http.HttpClient;
import org.openqa.selenium.remote.http.HttpClient.Factory;
import org.openqa.selenium.remote.internal.ApacheHttpClient;

public class ProxiedRemoteExample {
    private static final String PROXY_HOST = "localhost";
    private static final int PROXY_PORT = 8888;

    public ProxiedRemoteExample() throws MalformedURLException {
        InternetExplorerOptions ieOptions = new InternetExplorerOptions();
        RemoteWebDriver driver = new RemoteWebDriver(new HttpCommandExecutor(new HashMap<String, CommandInfo>(),
                new URL("http://localhost:5555/"), new Factory() {
                    private HttpClientBuilder builder;
                    {
                        builder = HttpClientBuilder.create();
                        builder.setProxy(new HttpHost(PROXY_HOST, PROXY_PORT));
                    }
                    @Override
                    public HttpClient createClient(URL url) {
                        return new ApacheHttpClient(builder.build(), url);
                    }

                }), ieOptions);
    }

}

score 0 · Answer 4 · answered Aug 30 '21 at 14:04

With org.seleniumhq.selenium:selenium-java:4.0.0-beta-3 I had to apply proxy settings in the following way:

configure async http client to use proxy settings
- create ahc.properties file under org/asynchttpclient/config folder
- file content: org.asynchttpclient.useProxyProperties = true
configure JVM proxy properties
- System.getProperties().setProperty("http.proxyHost", "yourProxyHost")
- System.getProperties().setProperty("http.proxyPort", "yourProxyPort")

score 0 · Answer 5 · edited Apr 03 '23 at 17:54

After v4.0.0-alpha-7, the OkHttp implementation of the http client has been removed.

But I found an easy way to set the proxy after rereading Andrew's answer:

var url = new URL("Your grid endpoint like browserstack or saucelabs");
var proxy = new java.net.Proxy(java.net.Proxy.Type.HTTP,
                    new InetSocketAddress("Your proxy", 8080));
var executor = new HttpCommandExecutor(ClientConfig.defaultConfig().baseUrl(url).proxy(proxy));
var driver = new RemoteWebDriver(executor, browserOptions);

Using Selenium RemoteWebDriver behind corporate proxy

5 Answers5

Linked