2

I'm trying to handle HTTPS connections from web browsers in decrypted data using Python SSL socket. I thought I can do this like a man-in-the-middle attack via my Python proxy server program, by pretending to be an HTTPS server, and I've already generated my keys using OpenSSL. However, browser treats my program as a proxy server, not as an HTTPS server. How can my program pretend to be an HTTPS server and specify my own keys to communicate with the browser?

I'm using Chrome and a Chrome extension SwitchySharp to forward requests to another local port (HTTP to 127.0.0.1:8080, and HTTPS to 127.0.0.1:8081). It works well for HTTP requests, but doesn't work for HTTPS.

Background Knowledge

When handling HTTPS with a proxy server, the browser first sends a CONNECT request to proxy:

CONNECT xxx.xxx:443 HTTP/1.1
Host: xxx.xxx:443
Proxy-Connection: keep-alive
(some other headers)

and proxy creates a tunnel to target server, and then browser sends encrypted data via this tunnel. Therefore, the proxy server never gets decrypted data.

To get decrypted data, I've tried two ways.

First, I tried ssl.wrap_socket():

import socket
import ssl
import thread
def handle(conn):
    request = conn.recv(4096)
    print request
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind(('127.0.0.1', 8081))
sock.listen(10)
sock = ssl.wrap_socket(sock, 'pkey.pem', 'cert.pem', True)
while True:
    conn, addr = sock.accept()
    thread.start_new_thread(handle, (conn,))

But I got the following error:

Traceback (most recent call last):
  File "https.py", line 12, in <module>
    conn, addr = sock.accept()
  File "C:\Python27\lib\ssl.py", line 898, in accept
    server_side=True)
  File "C:\Python27\lib\ssl.py", line 369, in wrap_socket
    _context=self)
  File "C:\Python27\lib\ssl.py", line 617, in __init__
    self.do_handshake()
  File "C:\Python27\lib\ssl.py", line 846, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: HTTPS_PROXY_REQUEST] https proxy request (_ssl.c:726)

The cause of this error is that the CONNECT request is not encrypted so ssl can't handle it. I have no idea how to get rid of this error.

Then, I tried the following code:

import socket
import thread
def handler(conn, tunn):
    while True:
        data = conn.recv(40960)
        if data:
            tunn.sendall(data)
        else:
            break
def handle(conn):
    request = conn.recv(40960)
    print request
    i = request.find(' ') + 1
    j = request.find(' ', i)
    host, port = request[i : j].split(':')
    port = int(port)
    tunn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    tunn.connect((host, port))
    conn.sendall('HTTP/1.1 200 Connection Established\r\n\r\n')
    thread.start_new_thread(handler, (conn, tunn))
    thread.start_new_thread(handler, (tunn, conn))
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind(('127.0.0.1', 8081))
sock.listen(10)
while True:
    conn, addr = sock.accept()
    thread.start_new_thread(handle, (conn,))

It worked (without ssl module), but I can't specify my own keys and therefore can't get decrypted data.

It seems that my program has to pretend to be an HTTPS server to "directly" communicate with browser, so I can specify my own key and get decrypted data. But I really have no idea how to pretend to be an HTTPS server and specify my own keys.

Can anyone help me? Thanks in advance.

UPD: I finally solved this problem using ssl module.

I was wrong --- I thought that after connection established the browser sends encrypted data. But it doesn't. In fact, those unreadable data are just SSL handshake. So what I need to do is just to use ssl.wrap_socket() after connection established. Besides, I have to additionally call do_handshake() of the ssl-wrapped socket.

import socket
import ssl
import thread
def handle(tid, conn):
    request = conn.recv(40960)
    i = request.find(' ') + 1
    j = request.find(' ', i)
    if request[i : j].find(':') != -1:
        host, port = request[i : j].split(':')
    else:
        host, port = request[i : j], 80
    port = int(port)
    tunn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    tunn.connect((host, port))
    conn.sendall('HTTP/1.1 200 Connection Established\r\n\r\n')
    conn_s = ssl.wrap_socket(conn, keyfile = 'cert/userkey.pem', certfile = 'cert/usercert.pem', server_side = True, do_handshake_on_connect = False)
    conn_s.do_handshake()
    data = conn_s.recv(40960)
    print data
    conn_s.sendall('HTTP/1.1 200 OK\r\n\r\n<h1>Hello, world!</h1>')
    conn_s.close()
    tunn.close()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind(('127.0.0.1', 8081))
sock.listen(10)
buf = {}
while True:
    conn, addr = sock.accept()
    thread.start_new_thread(handle, (conn,))

Anyway, it doesn't pretend to be an HTTPS server -- it still behaves as a proxy server. I just misunderstood those unreadable data.

Thanks to those who helped me in the comments.

Alljhatv
  • 21
  • 1
  • 4
  • Possible duplicate of this: https://stackoverflow.com/a/26851670/6529424 ? – eagle33322 Jan 29 '19 at 19:10
  • 2
    Possible duplicate of [Opening a SSL socket connection in Python](https://stackoverflow.com/questions/26851034/opening-a-ssl-socket-connection-in-python) – Yennefer Jan 29 '19 at 19:18
  • 1
    *"However, it seems that browser treats my program just as a proxy server."* - of course it does. You are using a browser extension which whole purpose is to let the browser switch between several **proxies**. In other words: the browser treats your program as a proxy because you've told the browser to treat your program as a proxy. – Steffen Ullrich Jan 29 '19 at 19:59
  • @SteffenUllrich Thanks for your explanation, and sorry for my poor English. It's not appropriate to use "seem" here. I have changed my expression. – Alljhatv Jan 29 '19 at 20:55
  • @Pier and eagle33322 Thanks for your help. I haved edited my question to clarify their difference. – Alljhatv Jan 29 '19 at 21:01
  • 1
    @Alljhatv: To be clear: there is no way that your server can make the browser use non-proxy requests. The browser decides what to do before it even contacts the server - and it decides to use proxy requests since you've explicitly configured a proxy. The best way is probably to handle proxy requests in your code. The other way is to redirect the normal traffic from your browser to your server with the help of a man in the middle attack - google for details but it it likely not possible to redirect it to some arbitrary port on localhost. – Steffen Ullrich Jan 29 '19 at 21:07
  • You will need to install your own trusted root certificate to intercept traffic this way, since the browser is explicitly designed to prevent this kind of interception. Even then, any websites with certificate pinning won’t work. I would recommend starting with an existing solution like mitmproxy or Charles. – Dietrich Epp Jan 29 '19 at 21:26
  • @SteffenUllrich That makes sense -- I'm supposed to deal with my browser instead of my code. I can try another Chrome extension to redirect traffic. Thanks for your help. – Alljhatv Jan 30 '19 at 07:23
  • @DietrichEpp Thanks for your advice. I'll install my root certificate on my computer later. I just want to decrypt the traffic and do something with them. Perhaps mitmproxy or Charles can help me understand how it works. – Alljhatv Jan 30 '19 at 07:30
  • @Alljhatv: it is unlikely you'll find one. The intended way to voluntarily redirect traffic from the browser side is a proxy - and not just sending non-proxy traffic somewhere else. As I said, if you insist of not making your program behave as a proxy you would need really man in the middle attacks like DNS spoofing, ARP spoofing etc which all work outside the browser. – Steffen Ullrich Jan 30 '19 at 07:30
  • @Alljhatv: The whole point point of encryption and PKI is to prevent exactly what you’re trying to do. Installing a new root certificate is not an optional step… you *have* to do it. It does not matter if you are using mitmproxy or Charles, or whether you are writing your own code. You still have to install a new root certificate. – Dietrich Epp Jan 30 '19 at 07:44

0 Answers0