What does Python's socket.recv() return for non-blocking sockets if no data is received until a timeout occurs?

Question

Basically, I've read in several places that socket.recv() will return whatever it can read, or an empty string signalling that the other side has shut down (the official docs don't even mention what it returns when the connection is shut down... great!). This is all fine and dandy for blocking sockets, since we know that recv() only returns when there actually is something to receive, so when it returns an empty string, it MUST mean the other side has closed the connection, right?

Okay, fine, but what happens when my socket is non-blocking?? I have searched a bit (maybe not enough, who knows?) and can't figure out how to tell when the other side has closed the connection using a non-blocking socket. There seems to be no method or attribute that tells us this, and comparing the return value of recv() to the empty string seems absolutely useless... is it just me having this problem?

As a simple example, let's say my socket's timeout is set to 1.2342342 (whatever non-negative number you like here) seconds and I call socket.recv(1024), but the other side doesn't send anything during that 1.2342342 second period. The recv() call will return an empty string and I have no clue as to whether the connection is still standing or not...

"(the official docs don't even mention what it returns when the connection is shut down... great!)" . Seven years later and still not fixed. This is so frustrating, spent almost an hour trying to figure out why my socket didn't raise an Error when the connection is closed. Then I was like ok let's set a timeout then. This also didn't raise an Exception and i was really confused. Thanks for your post, otherwise i would still be trying to figure this out. — KoKlA, Dec 01 '20 at 18:21
Is this problem still present in Python 3? I assume the OP is likely to have encountered this issue in Python 2, since the question was asked in 2013. — icedwater, Jul 21 '21 at 05:03
Ed of 2021, the python 3 documentation still does not really provide a clear description of return values for corrupted or dropped connections. Although the example code does look for b'' as a logical comparison. Thanks for the question! (and answers) :) https://docs.python.org/3/library/socket.html — Chemistpp, Nov 30 '21 at 20:34

mshildt · Accepted Answer · 2014-07-15T13:00:40.627

102

In the case of a non blocking socket that has no data available, recv will throw the socket.error exception and the value of the exception will have the errno of either EAGAIN or EWOULDBLOCK. Example:

import sys
import socket
import fcntl, os
import errno
from time import sleep

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1',9999))
fcntl.fcntl(s, fcntl.F_SETFL, os.O_NONBLOCK)

while True:
    try:
        msg = s.recv(4096)
    except socket.error, e:
        err = e.args[0]
        if err == errno.EAGAIN or err == errno.EWOULDBLOCK:
            sleep(1)
            print 'No data available'
            continue
        else:
            # a "real" error occurred
            print e
            sys.exit(1)
    else:
        # got a message, do something :)

The situation is a little different in the case where you've enabled non-blocking behavior via a time out with socket.settimeout(n) or socket.setblocking(False). In this case a socket.error is stil raised, but in the case of a time out, the accompanying value of the exception is always a string set to 'timed out'. So, to handle this case you can do:

import sys
import socket
from time import sleep

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1',9999))
s.settimeout(2)

while True:
    try:
        msg = s.recv(4096)
    except socket.timeout, e:
        err = e.args[0]
        # this next if/else is a bit redundant, but illustrates how the
        # timeout exception is setup
        if err == 'timed out':
            sleep(1)
            print 'recv timed out, retry later'
            continue
        else:
            print e
            sys.exit(1)
    except socket.error, e:
        # Something else happened, handle error, exit, etc.
        print e
        sys.exit(1)
    else:
        if len(msg) == 0:
            print 'orderly shutdown on server end'
            sys.exit(0)
        else:
            # got a message do something :)

As indicated in the comments, this is also a more portable solution since it doesn't depend on OS specific functionality to put the socket into non-blockng mode.

See recv(2) and python socket for more details.

edited Jul 15 '14 at 13:00

answered May 25 '13 at 01:26

mshildt

8,782
3
34
41

2

Okay, this is EXACTLY what I needed to know. Thank you! I wouldn't have needed to ask this question if only the official docs had mentioned that an exception is raised by this method though... a bit disappointed with the docs :( but very happy with your answer :) thumbs up! – El Ninja Trepador May 25 '13 at 01:54
2

Just a side note about your code. Setting the socket to non-blocking mode in a more Pythonic and cross-platform (and readable imho, without the C-style flags) way is to simply call `s.settimeout(whatever_nonnegative_number_of_your_liking)` – El Ninja Trepador May 25 '13 at 02:00
Actually, I am only now noticing this only answers half of my question. The thing is, when `s` has a timeout defined and `recv()` fails without data after a timeout, a `socket.timeout` exception is raised (not `socket.error`). The timeout exception still doesn't let me conclude anything about the state of the connection. I am supposing `socket.error` is raised if the connection is closed though. Can anyone confirm this? – El Ninja Trepador May 25 '13 at 02:12
1

Updated the examples to address your comment about non-blocking behavior with s.settimeout(). You're correct in that the situation is different there. – mshildt May 25 '13 at 14:12
1

There python socket.setblocking() method to make socket being non-blocking – Zaar Hai Jul 15 '14 at 08:57
@ZaarHai Thanks for pointing that out. I've added mention of it before the second code example. – mshildt Jul 15 '14 at 13:02
The first example does not seem to work for me: http://stackoverflow.com/questions/37330993/sock-recv-returns-empty-string-when-connection-is-dead-on-non-blocking-socke – replay May 19 '16 at 18:14
So I can see that OS socket non-blocking is not the same as Python socket non-blocking. However can you expand on where does `select.select` or `select.poll` fit in the above examples. Wouldn't it essentially convert the above busy polling loop into a https://en.wikipedia.org/wiki/Asynchronous_I/O#Select.28.2Fpoll.29_loops which can involve sleeping the thread for the OS to reschedule another thread? – CMCDragonkai Oct 15 '16 at 15:39
You maybe meant "socket.settimeout(n) or socket.setblocking(True)" before second code block? – Mikhail M Apr 26 '22 at 14:41

score 10 · Answer 2 · answered May 25 '13 at 01:43

10

It is simple: if recv() returns 0 bytes; you will not receive any more data on this connection. Ever. You still might be able to send.

It means that your non-blocking socket have to raise an exception (it might be system-dependent) if no data is available but the connection is still alive (the other end may send).

answered May 25 '13 at 01:43

jfs

399,953
195
994
1,670

Thanks for the answer. Nice and simple. I read the docs but my brain seemed to ignore that part somehow :D Finally, after actually thinking about it for a bit, I noticed that `recv()` will never return an empty string unless the connection has been broken, since in non-blocking mode `recv()` will raise `socket.timeout` when no data is available during the timeout period. Thanks again! :) – El Ninja Trepador May 26 '13 at 16:14

Barış Uşaklı · Answer 3 · 2013-05-25T01:11:36.470

When you use recv in connection with select if the socket is ready to be read from but there is no data to read that means the client has closed the connection.

Here is some code that handles this, also note the exception that is thrown when recv is called a second time in the while loop. If there is nothing left to read this exception will be thrown it doesn't mean the client has closed the connection :

def listenToSockets(self):

    while True:

        changed_sockets = self.currentSockets

        ready_to_read, ready_to_write, in_error = select.select(changed_sockets, [], [], 0.1)

        for s in ready_to_read:

            if s == self.serverSocket:
                self.acceptNewConnection(s)
            else:
                self.readDataFromSocket(s)

And the function that receives the data :

def readDataFromSocket(self, socket):

    data = ''
    buffer = ''
    try:

        while True:
            data = socket.recv(4096)

            if not data: 
                break

            buffer += data

    except error, (errorCode,message): 
        # error 10035 is no data available, it is non-fatal
        if errorCode != 10035:
            print 'socket.error - ('+str(errorCode)+') ' + message


    if data:
        print 'received '+ buffer
    else:
        print 'disconnected'

This is the situation that I have faced; We use select.poll and I was wondering if the poll would return the client socket to indicate read, if , the socket has got closed ? I guess this was the question also and your answer is indicating that, poll will select the socket for read, and I can do socket.recv and check if there is any data to check if connection is closed — Alex Punnen, May 01 '16 at 12:17

score 2 · Answer 4 · answered May 25 '13 at 05:46

2

Just to complete the existing answers, I'd suggest using select instead of nonblocking sockets. The point is that nonblocking sockets complicate stuff (except perhaps sending), so I'd say there is no reason to use them at all. If you regularly have the problem that your app is blocked waiting for IO, I would also consider doing the IO in a separate thread in the background.

answered May 25 '13 at 05:46

Ulrich Eckhardt

16,572
3
28
55

5

Having your thread block (perhaps indefinitely) when you need it to be doing something else can also complicate things... and it's possible (at least under Linux) for a socket operation to block even after select() indicated that it would not :( – Jeremy Friesner Nov 03 '14 at 17:48

What does Python's socket.recv() return for non-blocking sockets if no data is received until a timeout occurs?

4 Answers4

Linked