How to connect() with a non-blocking socket?

Question

In Python, I would like to use socket.connect() on a socket that I have set to non-blocking. When I try to do this, the method always throws a BlockingIOError. When I ignore the error (as below) the program executes as expected. When I set the socket to non-blocking after it is connected, there are no errors. When I use select.select() to ensure the socket is readable or writable, I still get the error.

testserver.py

import socket
import select

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setblocking(0)

host = socket.gethostname()
port = 1234

sock.bind((host, port))
sock.listen(5)

while True:
    select.select([sock], [], [])
    con, addr = sock.accept()
    message = con.recv(1024).decode('UTF-8')
    print(message)

testclient.py

import socket
import select

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setblocking(0)

host = socket.gethostname()
port = 1234

try:
    sock.connect((host, port))
except BlockingIOError as e:
    print("BlockingIOError")

msg = "--> From the client\n"

select.select([], [sock], [])
if sock.send(bytes(msg, 'UTF-8')) == len(msg):
    print("sent ", repr(msg), " successfully.")

sock.close()

terminal 1

$ python testserver.py
--> From the client

terminal 2

$ python testclient.py
BlockingIOError
sent  '--> From the client\n'  successfully.

This code works correctly except for the BlockingIOError on the first connect(). The documentation for the error reads like this: Raised when an operation would block on an object (e.g. socket) set for non-blocking operation.

How do I properly connect() with a socket set to non-blocking? Can I make connect() non-blocking? Or is it appropriate to just ignore the error?

Answer 1

When using socket.connect with a non-blocking socket, it is somewhat expected to get a BlockingIOError at first. See TCP Connect error 115 Operation in Progress What is the Cause? for an explanation of the cause. Basically, the socket isn't ready yet and raises BlockingIOError: [Errno 115] Operation now in progress, also known as EINPROGRESS.

The solution is to either catch and ignore the exception or to use socket.connect_ex instead of socket.connect because that method doesn't raise an exception. Especially note the last sentence from its description in the Python docs:

socket.connect_ex(address)

Like connect(address), but return an error indicator instead of raising an exception for errors returned by the C-level connect() call (other problems, such as “host not found,” can still raise exceptions). The error indicator is 0 if the operation succeeded, otherwise the value of the errno variable. This is useful to support, for example, asynchronous connects.

Source: https://docs.python.org/3/library/socket.html#socket.socket.connect_ex

If you want to keep using socket.connect, you can catch and ignore the responsible EINPROGRESS error:

>>> import socket
>>> 
>>> # bad
>>> s = socket.socket()
>>> s.setblocking(False)
>>> s.connect(("127.0.0.1", 8080))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
BlockingIOError: [Errno 115] Operation now in progress
>>> 
>>> # good
>>> s = socket.socket()
>>> s.setblocking(False)
>>> try:
...     s.connect(("127.0.0.1", 8080))
... except OSError as exc:
...     if exc.errno != 115:  # EINPROGRESS
...         raise
... 
>>> 

Answer 2

The trick here is that when the select completes the first time, then you need to call sock.connect again. The socket is not connected until you have received a successful return status from connect.

Just add these two lines after the first call to select completes:

print("first select completed")
sock.connect((host, port))

EDIT:
Followup. I was wrong to have stated that an additional call to sock.connect is required. It is however a good way to discover whether the original non-blocking call to connect succeeded if you wish to handle the connection failure in its own code path.

The traditional way of achieving this in C code is explained here: Async connect and disconnect with epoll (Linux)

This involves calling getsockopt. You can do this in python too but the result you get back from sock.getsockopt is a bytes object. And if it represents a failure, you then need to convert it into an integer errno value and map that to a string (or exception or whatever you require to communicate the issue to the outside world). Calling sock.connect again maps the errno value to an appropriate exception already.

Solution 2: You can also simply defer calling sock.setblocking(0) until after the connect has completed.