21

I'm using urllib.request.urlretrieve to download a file to local.

urllib.request.urlretrieve(url_string,file_name)

It throws error:

ssl.CertificateError was unhandled by user code Message: hostname 'foo.net' doesn't match either of 'a248.e.akamai.net', '.akamaihd.net', '.akamaihd-staging.net', '.akamaized.net', '.akamaized-staging.net'

If you copy the url into Chrome, it will show you a notification and you need to say something like "keep going to the url".

falsetru
  • 357,413
  • 63
  • 732
  • 636
Bing Gan
  • 415
  • 3
  • 6
  • 12
  • @JoranBeasley, [`urllib.request.urlretrieve`](https://docs.python.org/3/library/urllib.request.html#urllib.request.urlretrieve) does not accept `verify` keyword argument. Maybe you're confused with [`requests` library](https://python-requests.org). – falsetru Nov 18 '15 at 01:31

2 Answers2

49

Use urllib.request.urlopen with custom ssl context:

import ssl
import urllib.request

ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

with urllib.request.urlopen(url_string, context=ctx) as u, \
        open(file_name, 'wb') as f:
    f.write(u.read())

Alternatively, if you use requests library, it could be simpler:

import requests

with open(file_name, 'wb') as f:
    resp = requests.get(url_string, verify=False)
    f.write(resp.content)
falsetru
  • 357,413
  • 63
  • 732
  • 636
  • 1
    I guess there are some typo for the first solution, This works fine :D with urllib.request.urlopen(url_string, context=ctx) as u: f = open(file_name, 'wb') f.write(u.read()) Not sure if we have requests library in Python3 tho – Bing Gan Nov 18 '15 at 02:16
  • @BingGan, I fixed the typo. Thank you for your feedback. `requests` supports Python 3.x. `pip install requests` and benefit. :) – falsetru Nov 18 '15 at 02:29
  • I am getting `Cannot set verify_mode to CERT_NONE when check_hostname is enabled.` even after setting `check_hostname=False`. Any idea? – viveksinghggits Dec 25 '18 at 15:14
  • ok, so we have to set the `check_hostname` before the `verify_mode` – viveksinghggits Dec 25 '18 at 15:27
  • 1
    `shutil.copyfileobj(u, f)` could be used instead of `f.write(u.read())`, to avoid loading the whole content into memory. – jfs Aug 05 '21 at 17:29
6

Function urllib.request.urlretrieve doesn't accept any SSL options but urllib.request.urlopen does.

However instead creating a secure SSL context with ssl.create_default_context() and making it insecure you can create an insecure context with ssl.SSLContext():

This:

ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

is equivalent to:

ctx = ssl.SSLContext()

(For Python < 3.5.3 use ssl.SSLContext(ssl.PROTOCOL_TLSv1))

Which makes a nice one-liner:

import ssl
import urllib.request

with urllib.request.urlopen("https://wrong.host.badssl.com/", context=ssl.SSLContext()) as url:
    print(url.read())
mx0
  • 6,445
  • 12
  • 49
  • 54
  • `ctx` (the first version) gives UNSUPPORTED PROTOCOL. – James Hirschorn Aug 30 '20 at 18:32
  • @JamesHirschorn that do you mean? How you get that error? – mx0 Aug 31 '20 at 08:14
  • I got that error using your code, with the first definition of ctx, and a different url. So: with urllib.request.urlopen(my_url, context=ctx) as url: print(url.read()) I guess the website might not support a recent protocol, even if the certificate verification is off? – James Hirschorn Aug 31 '20 at 14:48
  • Yes, this trick is to disable specific checks of a certificate. Still protocol and ciphers have to match. – mx0 Sep 01 '20 at 11:24