2

I have a python app that connects to a web page (wikis.xxx.yyy) via HTTPS to retrieve and parse data from it. I can browse to the web page given that the certificate chain required to do so is in the browser (my system uses Ubuntu 14.04 LTS).

$ python --version
Python 2.7.6

The call to connect:

data = requests.post(URL, data=payload)

is failing with the following error: 

requests.exceptions.SSLError: [Errno bad handshake] [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]

I can't use verify=False given that if I do I get this error:

/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)

I tried passing the certificate in the verify as a client side side certificate (http://docs.python-requests.org/en/latest/user/advanced/): data = requests.post(radio_status_URL, data=payload, verify="/home/user/.cert/foo.pem") but it does not connect and actually does not return any errors.

Thinking that I needed to install the certificate chain into the trusted Ubuntu CA store I followed the instructions from: https://askubuntu.com/questions/73287/how-do-i-install-a-root-certificate The foo.cert certificate that I added has the following format:

-----BEGIN CERTIFICATE-----
<....>
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
<....>
-----END CERTIFICATE-----

Using openssl I can see that the certificate chain is OK and that I can connect to wikis.xxx.yyy :-)

$ openssl s_client -CApath ~/.cert/ -connect wikis.xxx.yyy:443
CONNECTED(00000003)
depth=2 DC = local, DC = windows, CN = RIM Root CA MCA01YKF
verify return:1
depth=1 DC = net, DC = rim, CN = RIM Subordinate CA MCA03YKF
verify return:1
depth=0 C = CA, ST = Ontario, L = Waterloo, O = BlackBerry, OU = BTS Operations, CN = wikis.xxx.yyy, emailAddress = no-reply@xxx.yyy

verify return:1
---
Certificate chain
 0 s:/C=CA/ST=Ontario/L=Waterloo/O=BlackBerry/OU=BTS Operations/CN=wikis.xxx.yyy/emailAddress=no-reply@xxx.yyy
   i:/DC=net/DC=rim/CN=RIM Subordinate CA MCA03YKF
---
Server certificate
-----BEGIN CERTIFICATE-----
<...>
-----END CERTIFICATE-----
subject=/C=CA/ST=Ontario/L=Waterloo/O=BlackBerry/OU=BTS Operations/CN=wikis.xxx.yyy/emailAddress=no-reply@xxx.yyy
issuer=/DC=net/DC=rim/CN=RIM Subordinate CA MCA03YKF
---
No client certificate CA names sent
---
SSL handshake has read 1810 bytes and written 609 bytes
---
New, TLSv1/SSLv3, Cipher is RC4-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : RC4-SHA
    Session-ID: FFD3D3EAC3807427D2D5E5BB2D70F064E2BB14218A368676A3225FBC5E5F1078
    Session-ID-ctx: 
    Master-Key: 930FBFB110BFC861E06C6A27DEBB15C2524A86F7EFC56693F4E52BA33F0AD236E5182169039909EC1BB3DE89C4C96B2B
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1442413894
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)

So it works and gives a ret code of 0. Unfortunately python fails to do the same and I have no clue how to connect :-(

How do I fix this?

Community
  • 1
  • 1
Tomas
  • 944
  • 8
  • 10
  • Just to clarify something: verify=False does not actually fail to return information back from the web page (in spite of the warning) but is highly discouraged due to Man in the middle attacks. – Tomas Sep 16 '15 at 16:17
  • Does `r=requests.post("https://httpbin.org/post")` work correctly for you? – Robᵩ Sep 16 '15 at 16:39
  • It did return something: { "args": {}, "data": "", "files": {}, "form": { "os_password": "YYYYYYY", "os_username": "XXXXXXX" }, "headers": { "Accept": "*/*", "Accept-Encoding": "gzip, deflate", "Content-Length": "43", "Content-Type": "application/x-www-form-urlencoded", "Host": "httpbin.org", "User-Agent": "python-requests/2.7.0 CPython/2.7.6 Linux/3.16.0-44-generic" }, "json": null, "origin": "208.93.78.35", "url": "https://httpbin.org/post" } – Tomas Sep 21 '15 at 19:03

1 Answers1

1

From your last testing case it shows that wikis.xxx.yyy use TLSv1.2, however Requests module needs a little revise to use it.

Please refer to this post: https://lukasa.co.uk/2013/01/Choosing_SSL_Version_In_Requests/

And there is a good plug-in to do that: https://toolbelt.readthedocs.org/en/latest/user.html#ssladapter

BTW, if you have installed urllib3, the Requests will use it, but urllib3 doesn't support TLS v1.2 until now.

Adrian
  • 76
  • 3