Python SSL Bad Handshake

Question

I'm having a very particular problem. In particular, I'd like to retrieve the content of the following website: https://www.mycardtamoil.it/

As I understood, this website has some issues with the SSL certificate. If you open it in Chrome, you have no issue, but if you try to retrieve the content via cURL (curl https://www.mycardtamoil.it/) you receive an error on ssl that can bypass using the option -k

When I move to Python3, I'm not able to bypass this issue; the code I've tested is:

import requests
response = requests.get('https://www.mycardtamoil.it/', verify=False)

but I get the following expection:

--------------------------------------------------------------------------- SysCallError Traceback (most recent call last) C:\ProgramData\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname) 440 try: --> 441 cnx.do_handshake() 442 except OpenSSL.SSL.WantReadError:

C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\SSL.py in do_handshake(self) 1906 result = _lib.SSL_do_handshake(self._ssl) -> 1907 self._raise_ssl_error(self._ssl, result) 1908

C:\ProgramData\Anaconda3\lib\site-packages\OpenSSL\SSL.py in _raise_ssl_error(self, ssl, result) 1631 raise SysCallError(errno, errorcode.get(errno)) -> 1632 raise SysCallError(-1, "Unexpected EOF") 1633 else:

SysCallError: (-1, 'Unexpected EOF')

During handling of the above exception, another exception occurred:

SSLError Traceback (most recent call last) C:\ProgramData\Anaconda3\lib\site-packages\urllib3\connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw) 600 body=body, headers=headers, --> 601 chunked=chunked) 602

C:\ProgramData\Anaconda3\lib\site-packages\urllib3\connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw) 345 try: --> 346 self._validate_conn(conn) 347 except (SocketTimeout, BaseSSLError) as e:

C:\ProgramData\Anaconda3\lib\site-packages\urllib3\connectionpool.py in _validate_conn(self, conn) 849 if not getattr(conn, 'sock', None): # AppEngine might not have .sock --> 850 conn.connect() 851

C:\ProgramData\Anaconda3\lib\site-packages\urllib3\connection.py in connect(self) 325 server_hostname=hostname, --> 326 ssl_context=context) 327

C:\ProgramData\Anaconda3\lib\site-packages\urllib3\util\ssl_.py in ssl_wrap_socket(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version, ciphers, ssl_context, ca_cert_dir) 328 if HAS_SNI: # Platform-specific: OpenSSL with enabled SNI --> 329 return context.wrap_socket(sock, server_hostname=server_hostname) 330

C:\ProgramData\Anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname) 447 except OpenSSL.SSL.Error as e: --> 448 raise ssl.SSLError('bad handshake: %r' % e) 449 break

SSLError: ("bad handshake: SysCallError(-1, 'Unexpected EOF')",)

During handling of the above exception, another exception occurred:

MaxRetryError Traceback (most recent call last) C:\ProgramData\Anaconda3\lib\site-packages\requests\adapters.py in send(self, request, stream, timeout, verify, cert, proxies) 439 retries=self.max_retries, --> 440 timeout=timeout 441 )

C:\ProgramData\Anaconda3\lib\site-packages\urllib3\connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw) 638 retries = retries.increment(method, url, error=e, _pool=self, --> 639 _stacktrace=sys.exc_info()[2]) 640 retries.sleep()

C:\ProgramData\Anaconda3\lib\site-packages\urllib3\util\retry.py in increment(self, method, url, response, error, _pool, _stacktrace) 387 if new_retry.is_exhausted(): --> 388 raise MaxRetryError(_pool, url, error or ResponseError(cause)) 389

MaxRetryError: HTTPSConnectionPool(host='www.mycardtamoil.it', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: SysCallError(-1, 'Unexpected EOF')",),))

During handling of the above exception, another exception occurred:

SSLError Traceback (most recent call last) in () 1 import requests ----> 2 response = requests.get('https://www.mycardtamoil.it/', verify=False)

C:\ProgramData\Anaconda3\lib\site-packages\requests\api.py in get(url, params, **kwargs) 70 71 kwargs.setdefault('allow_redirects', True) ---> 72 return request('get', url, params=params, **kwargs) 73 74

C:\ProgramData\Anaconda3\lib\site-packages\requests\api.py in request(method, url, **kwargs) 56 # cases, and look like a memory leak in others. 57 with sessions.Session() as session: ---> 58 return session.request(method=method, url=url, **kwargs) 59 60

C:\ProgramData\Anaconda3\lib\site-packages\requests\sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json) 506 } 507 send_kwargs.update(settings) --> 508 resp = self.send(prep, **send_kwargs) 509 510 return resp

C:\ProgramData\Anaconda3\lib\site-packages\requests\sessions.py in send(self, request, **kwargs) 616 617 # Send the request --> 618 r = adapter.send(request, **kwargs) 619 620 # Total elapsed time of the request (approximately)

C:\ProgramData\Anaconda3\lib\site-packages\requests\adapters.py in send(self, request, stream, timeout, verify, cert, proxies) 504 if isinstance(e.reason, _SSLError): 505 # This branch is for urllib3 v1.22 and later. --> 506 raise SSLError(e, request=request) 507 508 raise ConnectionError(e, request=request)

SSLError: HTTPSConnectionPool(host='www.mycardtamoil.it', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: SysCallError(-1, 'Unexpected EOF')",),))

Can someone help me to understand how I can fix it using requests module?

Thanks in advance, Salvo

Answer 1

According to SSLLabs the server has a terrible setup. Notably it only supports TLS 1.0 and less (even the more or less broken SSLv3 and the absolutely broken SSLv2) and it also only supports ciphers which are considered weak or really insecure.

While clients usually support TLS 1.0 (still considered sufficiently secure despite known problems) they usually don't support weak and insecure ciphers - at least by default but sometimes they are not even compiled in any longer. Probably the less insecure cipher supported by the server is TLS_RSA_WITH_3DES_EDE_CBC_SHA (called DES-CBC3-SHA within OpenSSL).

Assuming that this cipher is still available in the version of OpenSSL used by your Python you could hack requests into enable this cipher. Based on this question this could be done like this:

import requests
requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS = 'DES-CBC3-SHA'
requests.get('https://www.mycardtamoil.it/')

While it now no longer gets a Unexpected EOF it gets another error: certificate verify failed. This is because the servers setup is not only broken regarding protocol versions and ciphers but also the certificate is not properly configured. The SSLLabs report shows among all the other problems also

This server's certificate chain is incomplete.

Adding verify=False like you did "fixes" the problem by adding even more insecurity, i.e. it skips the validation of the certificate. See here for how such situations can be properly fixed. Although, it is not clear if properly checking the certificate would actually significantly improve the security in this situation where you have to deal with a severely broken and insecure system. I recommend to not sent any sensitive data to this system, no matter if HTTPS is used or not.