65

I want to ignore the certification validation during my request to the server with an internal corporate link.

With python requests library I would do this:

r = requests.get(link, allow_redirects=False,verify=False)

How do I do the same with urllib2 library?

Cœur
  • 37,241
  • 25
  • 195
  • 267
Sangamesh Hs
  • 1,447
  • 3
  • 24
  • 39

6 Answers6

152

In the meantime urllib2 seems to verify server certificates by default. The warning, that was shown in the past disappeared for 2.7.9 and I currently ran into this problem in a test environment with a self signed certificate (and Python 2.7.9).

My evil workaround (don't do this in production!):

import urllib2
import ssl

ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

urllib2.urlopen("https://your-test-server.local", context=ctx)

According to docs calling SSLContext constructor directly should work, too. I haven't tried that.

Community
  • 1
  • 1
Enno Gröper
  • 4,391
  • 1
  • 27
  • 33
  • 6
    so evil and so working xD thanks for that. I already were fallen into despair. – Ramon Apr 06 '15 at 16:25
  • 4
    It seems that `ssl.create_default_context` is only available in Python 3.4+. – gosr Apr 14 '15 at 08:04
  • 1
    This works for me using python 2.7.9 (OSX Homebrew installed) Thanks! – checketts Apr 29 '15 at 19:49
  • 5
    This workaround doesn't work with either python 2.6.6 or 2.7.6. – Leonidas Tsampros May 08 '15 at 08:31
  • This work-around is perfect, but just don't call urllib2.urlopen() prior to 2.7.9 with the context parameter, that's all. I'm using this in both 2.7.10 and 2.6.x. Check version using this code: sys.version_info >= ( 2, 7, 9 ). – Russ Bateman Jul 31 '15 at 19:58
  • So... this was about to work for me, but I've also got to supply `HTTP Basic` authentication information at the same time. This technique seems to interfere with using `urllib2.install_opener(urllib2.build_opened(urllib2.HTTPBasicAuthHandler(mypwdmgr))`. Any suggestions for getting these working together? – Christopher Schultz Aug 27 '15 at 21:20
  • @ChristopherSchultz pass `urllib2.HTTPSHandler(context=ctx)` as additional argument to `urllib2.build_opener` – maxpolk Oct 15 '15 at 17:20
  • @maxpolk Yeah that's as clear as mud from the Python documentation, but I did in fact get it figured out a while back (sorry I didn't post back with an update). Since I needed to support optional authentication *and/or* hostname-verification-skipping, I ended up with a few nested conditionals to figure out what was happening and then calling `urllib2.install_opener(urllib2.build_opener(auth_handler, host_handler))` when I needed both. – Christopher Schultz Oct 15 '15 at 17:48
  • @eightx2 Do you know if there's anything like that on Python2.7? – m3nda May 22 '16 at 04:06
  • You say not to use this in production - what is it exactly that makes ignoring the cert validation so dangerous? Is it still dangerous if you know the URL you're hitting is safe? (e.g., in the original question the OP said they're hitting an internal corporate link, which will always be safe) – kibowki Nov 17 '17 at 21:57
  • I don't see the point of using encryption without verification. If you fully trust the network, why are you using encryption? In security I always think of the onion layer model. Every layer improves security and makes you depend less on the other layers. If you don't verify the endpoint, encryption is only producing heat. In my opinion most attackers, that can read your packets may probably modify them too. – Enno Gröper Nov 19 '17 at 07:21
  • This worked perfectly in python 2.7.13 using `urllib` (not urllib2, which I didn't try) `ssl.create_default_context` was in fact available for me. – Kasapo Apr 23 '19 at 23:20
67

The easiest way:

python 2

import urllib2, ssl

request = urllib2.Request('https://somedomain.co/')
response = urllib2.urlopen(request, context=ssl._create_unverified_context())

python 3

from urllib.request import urlopen
import ssl

response = urlopen('https://somedomain.co', context=ssl._create_unverified_context())
Kron
  • 1,968
  • 1
  • 15
  • 15
  • 2
    Make sure you have a later python version. The version shipped with ubuntu 14.04 does not support this method. – Martlark Mar 20 '17 at 03:21
  • 3
    This method works but had change import statement to `from urllib.request import urlopen` instead of `import urllib2`. See the accepted answer at https://stackoverflow.com/questions/2792650/import-error-no-module-name-urllib2 for more info. – aye2m Jul 05 '18 at 08:54
35

For those who uses an opener, you can achieve the same thing based on Enno Gröper's great answer:

import urllib2, ssl

ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

opener = urllib2.build_opener(urllib2.HTTPSHandler(context=ctx), your_first_handler, your_second_handler[...])
opener.addheaders = [('Referer', 'http://example.org/blah.html')]

content = opener.open("https://localhost/").read()

And then use it as before.

According to build_opener and HTTPSHandler, a HTTPSHandler is added if ssl module exists, here we just specify our own instead of the default one.

Damien
  • 767
  • 9
  • 13
  • Good call, this looks similar to the method described at this site. http://thejosephturner.com/blog/post/https-certificate-verification-in-python-with-urllib2/ – dragon788 Jul 11 '16 at 18:14
  • I disagree, IMHO, it's not similar. The article you mention uses a class extention while this method uses vanilla implementation and only ssl context. That said, the article you mentioned is a good point for whoever wants to alter and extend default HTTPSHandler behavior. – Damien Jul 11 '16 at 18:49
  • I see what you mean, I hadn't caught that nuance, yours is definitely cleaner without having to extend anything. – dragon788 Jul 14 '16 at 22:30
7

According to @Enno Gröper 's post, I've tried the SSLContext constructor and it works well on my machine. code as below:

import ssl
ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
urllib2.urlopen("https://your-test-server.local", context=ctx)

if you need opener, just added this context like:

opener = urllib2.build_opener(urllib2.HTTPSHandler(context=ctx))

NOTE: all above test environment is python 2.7.12. I use PROTOCOL_SSLv23 here since the doc says so, other protocol might also works but depends on your machine and remote server, please check the doc for detail.

Jkm
  • 187
  • 1
  • 2
  • 6
1

A more explicit example, built on Damien's code (calls a test resource at http://httpbin.org/). For python3. Note that if the server redirects to another URL, uri in add_password has to contain the new root URL (it's possible to pass a list of URLs, also). 

import ssl    
import urllib.parse
import urllib.request

def get_resource(uri, user, passwd=False):
    """
    Get the content of the SSL page.
    """
    uri = 'https://httpbin.org/basic-auth/user/passwd'
    user = 'user'
    passwd = 'passwd'

    context = ssl.create_default_context()
    context.check_hostname = False
    context.verify_mode = ssl.CERT_NONE

    password_mgr = urllib.request.HTTPPasswordMgrWithDefaultRealm()
    password_mgr.add_password(None, uri, user, passwd)

    auth_handler = urllib.request.HTTPBasicAuthHandler(password_mgr)

    opener = urllib.request.build_opener(auth_handler, urllib.request.HTTPSHandler(context=context))

    urllib.request.install_opener(opener)

    return urllib.request.urlopen(uri).read()
marw
  • 2,939
  • 4
  • 28
  • 37
  • Thanks for answering the python3 way with standard library. Preparing context worked for me very well. – JaXt0r Jan 30 '18 at 07:31
-2

urllib2 does not verify server certificate by default. Check this documentation.

Edit: As pointed out in below comment, this is not true anymore for newer versions (seems like >= 2.7.9) of Python. Refer the below ANSWER

Community
  • 1
  • 1
thavan
  • 2,409
  • 24
  • 32