Error accessing ECMWF database with python

Question

I am tryibg to access the ECMWF database and download a simple set of data with python. Despite I have followed the very simple instructions in the website: https://software.ecmwf.int/wiki/display/CKB/How+to+download+ERA-Interim+data+from+the+ECMWF+data+archive

I got the following error.

How can it be solved?

You can find below also the example code used

I am using python 3.6.5 on Windows 8

================== RESTART: C:\Users\Giulio\Desktop\test.py ==================
2018-05-21 12:08:29 ECMWF API python library 1.5.0
2018-05-21 12:08:29 ECMWF API at https://api.ecmwf.int/v1
Error contacting the WebAPI, retrying in 60 seconds ...
Error contacting the WebAPI, retrying in 60 seconds ...
Error contacting the WebAPI, retrying in 60 seconds ...
Error contacting the WebAPI, retrying in 60 seconds ...
Error contacting the WebAPI, retrying in 60 seconds ...
Error contacting the WebAPI, retrying in 60 seconds ...
Error contacting the WebAPI, retrying in 60 seconds ...
Error contacting the WebAPI, retrying in 60 seconds ...
Error contacting the WebAPI, retrying in 60 seconds ...
Error contacting the WebAPI, retrying in 60 seconds ...
Could not contact the WebAPI after 10 tries, failing !
Traceback (most recent call last):
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 1026, in _send_output
    self.send(msg)
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 964, in send
    self.connect()
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\ssl.py", line 814, in __init__
    self.do_handshake()
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Giulio\Desktop\test.py", line 19, in <module>
    "target": "test.nc"
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\site-packages\ecmwfapi\api.py", line 519, in retrieve
    c = APIRequest(self.url, "datasets/%s" % (dataset,), self.email, self.key, self.trace, verbose=self.verbose)
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\site-packages\ecmwfapi\api.py", line 391, in __init__
    user = self.connection.call("%s/%s" % (self.url, "who-am-i"))
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\site-packages\ecmwfapi\api.py", line 172, in wrapped
    raise last_error
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\site-packages\ecmwfapi\api.py", line 140, in wrapped
    return func(self, *args, **kwargs)
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\site-packages\ecmwfapi\api.py", line 273, in call
    res = opener.open(req)
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 526, in open
    response = self._open(req, data)
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 544, in _open
    '_open', req)
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "C:\Users\Giulio\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)>

EXAMPLE SCRIPT:

#!/usr/bin/env python
from ecmwfapi import ECMWFDataServer

server = ECMWFDataServer()
server.retrieve({
    "class": "ei",
    "dataset": "interim",
    "expver": "1",
    "stream": "oper",
    "type": "fc",
    "levtype": "sfc",
    "param": "167.128",
    "date": "2017-08-01/to/2017-08-03",
    "time": "00:00:00",
    "step": "3",
    "grid": "0.75/0.75",
    "area":"75/-20/10/60",
    "format":"netcdf",
    "target": "test.nc"
})

giulio · Answer 1 · 2018-05-21T17:57:08.397

I solved the issue by following this link: https://software.ecmwf.int/wiki/display/WEBAPI/Web-API+Troubleshooting#Web-APITroubleshooting-3.2.1.3WARNING:httplib2.URLErrorreceivedNone%3Curlopenerror[SSL:CERTIFICATE_VERIFY_FAILED]certificateverifyfailed

The problem may be that the SSL library used in the Python version doesn't look in the correct certificate path.

In my case, ca-certificates have to be updated

The QuoVadis Root CA 2 G3 is the Root CA for api.ecmwf.int and can be downloaded from https://www.quovadisglobal.com/QVRepository/DownloadRootsAndCRL.aspx and installed locally.

For Windows it can be done in the following steps:

1.Type mmc.exe on search bar of the windows menu

2.Go to File > Add / Remove Snap In > Select Certificates > Click Add > Click OK

3.Expand Certificates - Current User

4.Right Click on Trusted Root Certification Authorities > All Tasks > Import

5.Follow the steps

alternatively

certutil -addstore -f "ROOT" new-root-certificate.crt

In the command prompt

Plus One although I also needed to install `QuoVadis Global SSL ICA G2` which is the root certificate for `stream.ecmwf.int` — MarkJ, May 17 '19 at 16:14

score 0 · Answer 2 · answered Jan 22 '19 at 19:39

I had exactly the same problem.

My setup: Windows 10 & Python 3.6.

I solved it by directly modifying the source code (non-optimal solution, but that works). This is to ignore certificate validation.

Open: ~ /Lib/site-packages/ecmwfapi/api.py

At the start of the script, add:

import ssl

Adds at the beginning of the "_transfer" function:

ctx = ssl.create_default_context ()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

Then modify (always within the function): urlopen (req) by:

urlopen (req, context = ctx)

Source: python ignore certificate validation urllib2

Cheerio!

Error accessing ECMWF database with python

2 Answers2