Does buildout/easy_install/setup_tools verify SSL certificates?

Question

I'm trying to diagnose this error:

Getting distribution for 'zc.buildout<2dev'.
Got zc.buildout 1.7.1.
Generated script '/opt/mytardis/releases/a549cd05272afe8f16c2fe5efe8158490acbde82/bin/buildout'.
Download error on http://pypi.python.org/simple/buildout-versions/: [Errno 104] Connection reset by peer -- Some packages may not be found!
Couldn't find index page for 'buildout-versions' (maybe misspelled?)
Download error on http://pypi.python.org/simple/: [Errno 104] Connection reset by peer -- Some packages may not be found!
Getting distribution for 'buildout-versions'.
STDERR: /usr/lib64/python2.6/distutils/dist.py:266: UserWarning: Unknown distribution option: 'src_root'
  warnings.warn(msg)
While:
  Installing.
  Loading extensions.
  Getting distribution for 'buildout-versions'.
Error: Couldn't find a distribution for 'buildout-versions'.

It happens deep inside a Chef + buildout installation stack. One thing I have discovered is that if I attempt to access the buildout-versions package directly:

$ wget https://pypi.python.org/packages/source/b/buildout-versions/buildout-versions-1.7.tar.gz#md5=731ecc0c9029f45826fa9f31d44e311d
--2013-07-09 12:50:18--  https://pypi.python.org/packages/source/b/buildout-versions/buildout-versions-1.7.tar.gz
Resolving proxy.redacted.com... 123.45.67.8
Connecting to proxy.redacted.com|123.45.67.8|:8080... connected.
ERROR: certificate common name “*.a.ssl.fastly.net” doesn’t match requested host name “pypi.python.org”.
To connect to pypi.python.org insecurely, use ‘--no-check-certificate’.

I can access the file fine from my desktop. So I suspect the proxy (provided by a university, and this server has to use it to reach the web). It's set with https_proxy=....

Is this the likely cause of buildout failing? Any way around it?

Can you access `http://pypi.python.org/simple/buildout-versions`? The buildout you use does *not* try to use HTTPS at all. — Martijn Pieters, Jul 09 '13 at 09:59
Hmm, this is getting complicated. Going to that site in a browser redirects to https:// . Going there in Lynx from the site gives SSL warnings. And most confusingly, wget on a different server using the same proxy works just fine. — Steve Bennett, Jul 10 '13 at 08:11

score 11 · Accepted Answer · answered Sep 18 '13 at 22:35

11

Your version of wget is too old.

wget started to support SNI (Server Name Indication) only since version 1.14 and that TLS extension is needed to be presented the correct certificate on pypi.python.org.

answered Sep 18 '13 at 22:35

Bruno Rohée

3,436
27
32

score 4 · Answer 2 · edited May 23 '17 at 12:30

4

Yes, zc.buildout and easy_install both use urllib2 to retrieve HTTPS resources, which does not verify SSL certificates:

Warning: HTTPS requests do not do any verification of the server’s certificate.

Your wget tool does verify certificates, but your local certificate authorities certificates are incomplete, it seems; see SSL certificate rejected trying to access GitHub over HTTPS behind firewall for instructions on how to update those.

As for your original error, it appears your firewall proxy is doing the peer resets.

As per PEP 476, Python 2.7.9 remedies this situation. From that version onwards, urllib2 will verify SSL certificates by default.

edited May 23 '17 at 12:30

Community

1
1

answered Jul 10 '13 at 09:26

Martijn Pieters

1,048,767
296
4,058
3,343

1

Which bit in particular is the "instructions on how to update those"? Tried `yum update openssl` and `yum install ca-certificates` (probably the wrong package name) but no good. – Steve Bennett Jul 11 '13 at 00:38
Pip does no longer use urllib2; it uses Requests now. – Piotr Dobrogost Sep 01 '14 at 09:43
@PiotrDobrogost: Ah, indeed, from 1.5 onwards in a change made in August 2013 I see. I removed `pip` from the list. – Martijn Pieters Sep 01 '14 at 09:52
Situation changes with 2.7.9/3.4.3, which by default perform a verification. – Dr. Jan-Philip Gehrcke Feb 04 '15 at 15:50
@Jan-PhilipGehrcke: thanks for the heads-up; I was just dealing with some [fallout from that change](https://github.com/pypa/pip/issues/2395), but I didn't realise I had answers on SO to update. – Martijn Pieters Feb 04 '15 at 16:15
@MartijnPieters: Thanks for updating. And, yes, this unfortunately is not the only system affected by the SSL refactoring in 2.7.9: https://github.com/gevent/gevent/issues/477 - http://bugs.python.org/issue22438 - https://github.com/boto/boto/issues/2901 – Dr. Jan-Philip Gehrcke Feb 04 '15 at 17:06

score 1 · Answer 3 · answered Feb 04 '15 at 15:49

Since Python 2.7.9 (released) / 3.4.3 (released soon), certificates are validated by default:

HTTPS certificate validation using the system's certificate store is now enabled by default. See PEP 476 for details.

https://www.python.org/downloads/release/python-279/

score 0 · Answer 4 · answered Oct 30 '13 at 03:50

0

you can try it:

wget http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg#md5=fe1f997bc722265116870bc7919059ea --no-check-certificate

answered Oct 30 '13 at 03:50

蔡正海

483
4
11

Seriously, telling people not to check certificates is bordering on the criminal... – Bruno Rohée Nov 20 '13 at 10:58
@BrunoRohée Hopefully authors of wget and other net tools do not agree with you :) – Piotr Dobrogost Feb 03 '14 at 10:59
There's a difference between selling a gun and telling someone to commit suicide, offering a --no-check-certificate is like selling a gun, telling someone to use it downloading a package from the great Internet is more akin to tell him to play russian roulette... – Bruno Rohée Feb 03 '14 at 17:31
instead of ignoring ssl errors, just use openssl(1) to get a copy of the certificate and add that to your CA store (ie /etc/pki/..,/etc/ssl/..) – Dwight Spencer Sep 17 '15 at 20:36

Does buildout/easy_install/setup_tools verify SSL certificates?

4 Answers4