Error when trying to connect to too many websockets

Question

so my code attempts to simultaneously send 300 GET requests and then connect to 300 websockets. It works with 3+ websockets/GET requests but once it gets in the high numbers it outputs an error. Also the GET requests are not being done asynchronously and the error only appears once they've all been sent.

This is the error:

[...] File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 748, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno 8] nodename nor servname provided, or not known

This is the code:

import websockets
import requests
import asyncio

async def get_order_book(symbol):
    r = requests.get(url='https://api.binance.com/api/v1/depth',
                     params={'symbol': symbol, 'limit': 20})
    r2 = requests.get(url='https://api.binance.com/api/v1/aggTrades',
                      params={'symbol': symbol, 'limit': 1})
    obdata = r.json()
    trdata = r2.json()

    print(symbol)

    ws_url = 'wss://stream.binance.com:9443/ws/' + lc_symbol + '@depth/' + lc_symbol + '@aggTrade'
    websocket = await websockets.connect(ws_url)

async def get_order_books():
    r = requests.get(url='https://api.binance.com/api/v1/ticker/24hr')
    await asyncio.gather(*[get_order_book(data['symbol']) for data in r.json()])

if __name__ == '__main__':
    asyncio.run(get_order_books())

Any idea why this happens?

Thanks!

Answer 1

Your code works as expected for me without error. Requests is synchronous which is why your code is run synchronously - get_order_book doesn't give up control until the websockets await.

If you want async requests see this answer:

https://stackoverflow.com/a/22414756/10840818

To use requests (or any other blocking libraries) with asyncio, you can use BaseEventLoop.run_in_executor to run a function in another thread and yield from it to get the result.