How can I set https.proxyHost and https.proxyPort for individual HttpsURLConnections?

Question

I'm trying to make HTTPS requests through a proxy. Here's what I've got so far, based on code from this question:

try {
    HttpsURLConnection connection = (HttpsURLConnection) new URL("https://proxylist.geonode.com/api/proxy-list?limit=1&page=1&sort_by=speed&sort_type=asc&protocols=https").openConnection();
    connection.setRequestMethod("GET");
    connection.setDoInput(true);
    connection.setDoOutput(true);
    connection.setRequestProperty("user-agent", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36");
    connection.setConnectTimeout(30000);
    connection.connect();
    
    BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
    String rawJSON = reader.readLine();
    if(rawJSON == null) throw new IOException("No data");

    JSONObject data = new JSONObject(rawJSON).getJSONArray("data").getJSONObject(0);

    String ipAddress = data.getString("ip"), port = data.getString("port");

    System.setProperty("https.proxyHost", ipAddress);
    System.setProperty("https.proxyPort", port);

    SSLContext sslContext = SSLContext.getInstance("SSL");

    // set up a TrustManager that trusts everything
    sslContext.init(null, new TrustManager[] { new X509TrustManager() {
        public X509Certificate[] getAcceptedIssuers() { return null; }
        public void checkClientTrusted(X509Certificate[] certs, String authType) {}
        public void checkServerTrusted(X509Certificate[] certs, String authType) {}
    } }, new SecureRandom());

    HttpsURLConnection.setDefaultSSLSocketFactory(sslContext.getSocketFactory());

    HttpsURLConnection.setDefaultHostnameVerifier((arg0, arg1) -> true);

    HttpsURLConnection testConnection = (HttpsURLConnection) new URL("https://example.com").openConnection();
    testConnection.connect();

    StringBuilder result = new StringBuilder();
    String line;
    try(BufferedReader reader2 = new BufferedReader(new InputStreamReader(testConnection.getInputStream()))) {
        while ((line = reader2.readLine()) != null) result.append(line);
    }

    System.out.println(result);
} catch(Exception e) {
    e.printStackTrace();
}

The code works, but there's a problem. My application (https://encyclosearch.org) is multithreaded, and I need to make some requests through a proxy, and some directly. Since system properties are global, if I set https.proxyHost and https.proxyPort using System.setProperty, some requests that aren't supposed to go through the proxy will go through the proxy.

I can use java.net.Proxy like this:

Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(ipAddress, Integer.parseInt(port)));

HttpsURLConnection testConnection = (HttpsURLConnection) new URL("http://example.com").openConnection(proxy);

But that only works for HTTP proxies, not HTTPS ones, so I can't make HTTPS requests. There's no Proxy.Type.HTTPS.

Any help would be greatly appreciated. Thanks in advance.

score 1 · Answer 1 · answered Dec 05 '21 at 22:57

If you choose which connections go through a proxy and which do not by the destination url then you can use the property http.nonProxyHosts. This property is used for http and https as the documentation states:

for the "non proxy hosts" list, the HTTPS protocol handler will use the same as the http handler (i.e. http.nonProxyHosts).

You set the property value by adding patterns of urls separated by | For example:

System.setProperty("http.nonProxyHosts", ”localhost|host.example.com”)

You could also use ProxySelector class for choosing which connections go through a proxy. for more info (a bit old): https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html

As for the absence of Proxy.Type.HTTPS, it is because a proxy in general is not the final destination so the secure connection will be with the final destination not the proxy itself. There is such thing as SSL tunneling through a proxy, but i am not well informed about it.

Thanks for your answer! The article was very helpful. – Henry Sanger Dec 05 '21 at 23:29 — Henry Sanger, Dec 05 '21 at 23:29

Henry Sanger · Accepted Answer · 2021-12-06T01:12:00.240

With @Bashi's help, I figured it out. For direct connections, I used:

url.openConnection(Proxy.NO_PROXY);

This works for Jsoup, too:

Document document = Jsoup.connect("https://example.com").proxy(Proxy.NO_PROXY).get();

Explanation from https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html:

Now, this guarantees you that this specific URL will be retrieved though a direct connection bypassing any other proxy settings, which can be convenient.

How can I set https.proxyHost and https.proxyPort for individual HttpsURLConnections?

2 Answers2