0

First: I have already read the two pages of answers from this thread around the same error message.

I couldn't fix it yet though! I've (re)installed the certificates, but it keeps returning this error whenever I try to import CSV files from a certain website.

urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)>

So, except creating a workaround to NOT verify certificates, and given that I've already installed the certificates on my macOS, what else could I try to do to fix it?

42piratas
  • 555
  • 1
  • 9
  • 26
  • 1
    *"... from a certain website. ..."* - if this problem is restricted to a specific website then the problem is likely caused by the site and not by your code. Typical problems are self-signed certificates (browser will complain too) or missing chain certificates (browser most times work around this misconfiguration, but tools like curl will fail too). More can be said if the site is known. Alternatively you might check the site with [SSLLabs](https://www.ssllabs.com/ssltest/analyze.html) yourself. – Steffen Ullrich Sep 28 '20 at 03:59
  • Website is cryptodatadownload.com. And ssllabs.com says "Trust: Yes". Thanks for the help anyway – 42piratas Sep 29 '20 at 17:11
  • 1
    The SSLLabs report for the website (which is www.cryptodatadownload.com and not simply cryptodatadownload.com) clearly says **This server's certificate chain is incomplete**. This is the reason for the problem you see - as I've predicted already in my first comment. Duplicate to (among others) [Python requests SSL error - certificate verify failed](https://stackoverflow.com/questions/46604114/python-requests-ssl-error-certificate-verify-failed). You have to include the missing chain certificates into your trust store. – Steffen Ullrich Sep 29 '20 at 17:56
  • @steffen-ullrich Would you know where my 'trust store' is in my local macOS. I couldn't find any answer to that yet in order to add the website certificate to my env. – 42piratas Oct 01 '20 at 22:02

1 Answers1

1

What is your Python version and OS?

This error means that your Python app fails to load CA certificate that signed the certificate of the server.

There are two cases:

  1. The website is signed by an unknown CA. Then you need to get the certificate of that CA and install to your system. To test this issue try to connect with your app to a well known site. E.g. stackoverflow.com. If it works, then system certificates are loaded, and only the certificate of the server's CA is missing.

  2. If your app can't verify the certificates of well know apps, it means that it loaded no CA certificates at startup. You should load them manually.

To load CA certificates in Python use SSLContext.load_verify_locations method.

UPDATE

Root of all evil

Open the site https://www.cryptodatadownload.com/ in Firefox. And enjoy the security threat page with the core reason Error code: SEC_ERROR_UNKNOWN_ISSUER

The problem is that this site's certificate is signed by GeoTrust RSA certificate, that is not included in Mozilla CA Store. Why Mozilla? Because it's CA certificate store is used by many open source projects.

An extra layer of complexity: since Python 3.8 it no longer uses certificates from Mac OS. You need to load CA certificates manually.

Solution

The simplest way is to extract the CA certificates for the target site using browser and load them in urlopen call or SSL context construction.

Here is the file with the CA certificate chain for https://www.cryptodatadownload.com/. I built it using Chrome on Windows.

Save it under name geotrust.cer and try the following snippet:

import urllib.request as request

request.urlopen("https://www.cryptodatadownload.com/", cafile="geotrust.cer")

On my python 3.6 it works fine.

More tricky path is to add those two certificates to the CA path of your Python installation, but it is VERY platform dependent. I can't tell you a receipt for Mac OS

Pak Uula
  • 2,750
  • 1
  • 8
  • 13
  • (1) I'm using Python 3.8 on a macOS. (2) I'm running the project within a venv though and wondering now if I have to activate the certificates within the venv somehow, even I've already executed '''/applications/Python3.8/Install Certificates.command'''. Do you know if I do? (3) For the target website, this is what ssllabs says: https://www.ssllabs.com/ssltest/analyze.html?d=www.cryptodatadownload.com Thanks for you help, appreciate it! – 42piratas Sep 29 '20 at 17:14
  • Thank you @pak-uula! I really appreciate your very thoughtful replies I've just tried to update the code according to your recommendation, but you were right about the trickiest part: I have no idea where to place 'geotrust.cer' file. I've tried to place in a few different places, but every time I run the code, it says it cannot find the 'geotrust.cer' – 42piratas Oct 01 '20 at 21:59
  • 1
    @anquadros: The place for the file does not matter. It matters that it contains the correct certificates and that you actually specify the place you use in the call of `urlopen` with the parameter `cafile`. – Steffen Ullrich Oct 02 '20 at 04:02
  • 1
    @anquadros, argument `cafile` is a path in your file system. For testing place it to your home directory and use the absolute path, e.g. `request.urlopen("https://www.cryptodatadownload.com/", cafile="/Users/anquadros/geotrust.cer")`. Where to put it for deployment is a [different story](https://docs.python.org/3.8/distutils/setupscript.html#installing-additional-files) – Pak Uula Oct 02 '20 at 11:21