1

Normally, when using httplib (python 2.x) or http.client (python 3.x), you set up a connection and make a request like so:

from http.client import HTTPConnection
from urllib.parse import urlparse, urlencode

url = urlparse("http://final.destination.example.com:8888/")
conn = HTTPConnection(url.netloc)
request_path = "%s?%s" % (url.path, url.query)
conn.request("POST", request_path, post_data)

(note that this example code has python 3.x imports)

I am in a situation where I would like to use http.client for performance reasons, but I'd also like to use a SOCKS proxy for remote access into a cluster.

Is there any way to use a SOCKS proxy with httplib/http.client?

user3617786
  • 121
  • 2
  • 5
  • I think it might be a duplicate, thanks @ImanMirzadeh I will cross post the answer below there so that that question has a complete answer (the reply stopped short of working code) – user3617786 Jul 03 '17 at 21:23

1 Answers1

2

I am answering my own question here so that hopefully the next person in this situation can find the solution.

First install PySocks with:

pip install PySocks

Then you need to manually set up your SOCKS proxy after making instantiating your HTTPConnection and informing it that it's going to be using a proxy:

from http.client import HTTPConnection
from urllib.parse import urlparse, urlencode

import socks

url = urlparse("http://final.destination.example.com:8888/")

conn = HTTPConnection('127.0.0.1', 9000) # use socks proxy address
conn.set_tunnel(url.netloc, url.port) # remote host and port that you actually want to talk to
conn.sock = socks.socksocket() # manually set socket
conn.sock.set_proxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9000) # use socks proxy address
conn.sock.connect((url.netloc, url.port)) # remote host and port that you actually want to talk to

request_path = "%s?%s" % (url.path, url.query)
conn.request("POST", request_path, post_data)
user3617786
  • 121
  • 2
  • 5