0

I'm trying to get a file via https using requests 2.11.1 with Python 2.7.12 and OpenSSL 1.0.2h (all installed from Anaconda) on MacOS 10.11.6 from behind a proxy. According to SSLLabs, the server supports TLS 1.0, 1.1, and 1.2. Moreover, I can successfully retrieve the file with wget (linked to OpenSSL 1.0.2h) if I explicitly set the secure-protocol to tlsv1 (but not if I set it to unsupported protocols like sslv2). However, if I try to explicitly set the secure protocol used by requests to TLSv1, TLSv1_1, or TLSv1_2, e.g., as follows, 

from requests_toolbelt import SSLAdapter
import requests
import ssl

s = requests.Session()
p = ssl.PROTOCOL_TLSv1
s.mount('https://', SSLAdapter(p))
r = s.get("https://anaconda.org/conda-forge/matplotlib/2.0.0b3/download/osx-64/matplotlib-2.0.0b3-np111py27_5.tar.bz2")

I encounter the following exception:

/Users/lebedov/anaconda2/lib/python2.7/site-packages/requests/adapters.pyc in send(self, request, stream, timeout, verify, cert, proxies)
    495         except (_SSLError, _HTTPError) as e:
    496             if isinstance(e, _SSLError):
--> 497                 raise SSLError(e, request=request)
    498             elif isinstance(e, ReadTimeoutError):
    499                 raise ReadTimeout(e, request=request)

SSLError: ("bad handshake: Error([('SSL routines', 'SSL3_GET_RECORD', 'wrong version number')],)",)

(Less surprisingly, explicitly setting the protocol to SSLv2, SSLv3, or SSLv23 also results in handshake exceptions.) Also, I don't observe any exception when attempting to get other sites via https. Any idea why the connection fails with requests even when I force it to use TLSv1?

lebedov
  • 1,371
  • 2
  • 12
  • 27
  • `https://myurl...` is not very helpful. Does SNI make a difference? Without SNI: `openssl s_client -connect : -tls1`. With SNI: `openssl s_client -connect : -tls1 -servername `. Also see [SSLv3 alert handshake failure with urllib2](http://stackoverflow.com/q/30918761), which is a Python 2.7 question. – jww Sep 07 '16 at 14:41
  • Explicit URI added. – lebedov Sep 07 '16 at 14:45
  • Where are you setting tls1? all you are doing is creating a variable p equal to ssl.PROTOCOL_TLSv1, you are not setting anything. What you need to do is this https://lukasa.co.uk/2013/01/Choosing_SSL_Version_In_Requests/ , without `s.mount('https://', some_adapter))` you may as well remove all the code bar r = ... – Padraic Cunningham Sep 07 '16 at 14:57
  • `openssl s_client` hangs behind the proxy, and the version I have doesn't appear to support the proxy option. In any event, the pyopenssl injection monkeypatch you alluded to doesn't appear to have any effect on the error; running the code snippet with Python 3.5.2 also resulted in an `SSLError` exception (with a message indicating that an "EOF occurred in violation of protocol"). – lebedov Sep 07 '16 at 14:58
  • @lebedov: since you are behind a proxy: I cannot see that you set any proxies in your code. Or do you get these from the environment? Which settings do you have there? And it is very unlikely that the proxy cares about the TLS version, except when it is a SSL intercepting proxy. – Steffen Ullrich Sep 07 '16 at 15:19
  • @PadraicCunningham: right - I neglected to paste that line into the code, but it was in the code I ran that produced the exception. Updated the code snippet accordingly. – lebedov Sep 07 '16 at 17:35
  • @SteffenUllrich: connections to the proxy are managed via cntlm; I therefore set HTTP_PROXY and HTTPS_PROXY to ``http://localhost:3128`` and ``https://localhost:3128``, respectively. Also, the exception doesn't occur when I try to access other sites via https (e.g., ``https://www.google.com``). – lebedov Sep 07 '16 at 17:49
  • @lebedov: If you could make a packet capture (i.e. wireshark) one could maybe see what exactly is going on, i.e. what the client sends and how the server/proxy is responding. And it would be good to have the successful connection with wget too for comparison. Please as pcap not hex. – Steffen Ullrich Sep 07 '16 at 18:00

1 Answers1

1

This issue proved to be due to the proxy rejecting HTTP headers with unrecognized user-agent strings. Explicitly setting the user-agent in the headers to something like Mozilla/4.0 (compatible; MSIE 5.5; Windows 98) via a hacked version of cntlm solved the problem.

lebedov
  • 1,371
  • 2
  • 12
  • 27