14

With the following code I get Cannot connect to host ...:443 ssl:True when I use the asynchronous aiohttp. When I use synchronous requests, it succeeds.

The whitehouse.gov links fail, but the google.com succeeds for both async and sync cases.

What is going wrong? This is with python 3.4.2 on FreeBSD8, aiohttp 0.14.4, requests 2.5.3

import asyncio
import aiohttp
import requests

urls = [
    'http://www.whitehouse.gov/cea/', 
    'http://www.whitehouse.gov/omb', 
    'http://www.google.com']


def test_sync():
    for url in urls:
        r = requests.get(url)
        print(r.status_code)


def test_async():
    for url in urls:
        try:
            r = yield from aiohttp.request('get', url)
        except aiohttp.errors.ClientOSError as e:
            print('bad eternal link %s: %s' % (url, e))
        else:
            print(r.status)


if __name__ == '__main__':
    print('async')
    asyncio.get_event_loop().run_until_complete(test_async())
    print('sync')
    test_sync()

The output from this is:

async
bad eternal link http://www.whitehouse.gov/cea: Cannot connect to host www.whitehouse.gov:443 ssl:True
bad eternal link http://www.whitehouse.gov/omb: Cannot connect to host www.whitehouse.gov:443 ssl:True
200
sync
200
200
200
Tim
  • 1,013
  • 3
  • 17
  • 36

4 Answers4

18

I suspect certificate validation chain is broken on your machine. On Ubuntu everything is working, as @dano mentioned.

Anyway, you may disable ssl validation by creating custom Connector instance:

import asyncio
import aiohttp

urls = [
    'http://www.whitehouse.gov/cea/',
    'http://www.whitehouse.gov/omb',
    'http://www.google.com']


def test_async():
    connector = aiohttp.TCPConnector(verify_ssl=False)
    for url in urls:
        try:
            r = yield from aiohttp.request('get', url, connector=connector)
        except aiohttp.errors.ClientOSError as e:
            print('bad eternal link %s: %s' % (url, e))
        else:
            print(r.status)


if __name__ == '__main__':
    print('async')
    asyncio.get_event_loop().run_until_complete(test_async())

BTW, requests library is shipped with own certificate bundle. Maybe we need to do the same for aiohttp?

UPD. See also https://github.com/aio-libs/aiohttp/issues/341

Andrew Svetlov
  • 16,730
  • 8
  • 66
  • 69
  • 2
    exactly the problem thanks. If I hadn't cleverly hidden behind the try/except, I would have gotten more information that would have helped: ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600). Your solution of turning off validation was the trick. – Tim Apr 27 '15 at 13:45
  • @Tim, full info for exception points to error source in exception chain, isnt it? – Andrew Svetlov Apr 27 '15 at 14:09
  • yes--without the try/except, all the proper info is shown. I was masking the errors with try/except and only got my exception text + "Can not connect to www.whitehouse.gov:443". If I had made the bare connection, I would have seen the full traceback--starting with ssl.SSLError and ending with aiohttp.errors.ClientOSError. lesson learned (don't mask exceptions when developing code!) – Tim Apr 27 '15 at 15:11
  • 3
    I commonly receive this same exception when setting `ssl=False` in `TCPConnector`. – Brad Solomon Jan 15 '19 at 22:54
5

I had the same problem on an old Linux server with out of date CA root certificates, and loading certifi CA certificate bundle in a SSLContext fixed the issue.

import aiohttp
import ssl
import certifi

ssl_context = ssl.create_default_context(cafile=certifi.where())
async with aiohttp.ClientSession() as session:
    async with session.get('https://some.foo/bar/', ssl=ssl_context) as response:
        print(await response.text())
Le Hibou
  • 1,541
  • 1
  • 10
  • 12
0

Don't use this answere as it might be equal to disabling certificate checks.

As pointed out by Le Hibou in the comments, ssl.Purpose.CLIENT_AUTH is meant for authenticating clients on the server side.

This value indicates that the context may be used to authenticate Web clients (therefore, it will be used to create server-side sockets).

Looking at the code for create_default_context() shows that certificate checks might be disabled or at least optional in this case.

I had the same error message (on Windows) and solved it with the following:

import aiohttp
import ssl


client = aiohttp.ClientSession()
client.post(
    'https://some.foo/bar/',
    json={"foo": "bar"},
    ssl=ssl.create_default_context(purpose=ssl.Purpose.CLIENT_AUTH))

This is using the ssl argument of request() to set a different SSL context than the default. The default is ssl.create_default_context().

The problem is that the default value for ssl.create_default_context() are:

ssl.create_default_context(purpose=Purpose.SERVER_AUTH, cafile=None, capath=None, cadata=None)

The Purpose.SERVER_AUTH doesn't seem to work when validating server certificates on the client side. Instead use Purpose.CLIENT_AUTH when validating as a client.

Tillmann Radmer
  • 139
  • 2
  • 8
  • This is confusing... According to [The Python Standard Library documentation](https://docs.python.org/3/library/ssl.html#ssl.Purpose.CLIENT_AUTH), `Purpose.CLIENT_AUTH` is for creating server-side sockets. – Le Hibou Mar 07 '19 at 14:47
  • Good point, I overlooked that. Looking at the source for `create_default_context()` it seems that certificate checking might be disabled or optional with `ssl.Purpose.CLIENT_AUTH`. – Tillmann Radmer Mar 13 '19 at 14:37
0

This works for me:

if sys.version_info[0] == 3 and sys.version_info[1] >= 8 and sys.platform.startswith('win'):
    asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
sap2me
  • 1
  • 1