Can send() on a TCP socket return >=0 and

Question

I've seen a number of questions regarding send() that discuss the underlying protocol. I'm fully aware that for TCP any message may be broken up into parts as it's sent and there's no guarantee that the receiver will get the message in one atomic operation. In this question I'm talking solely about the behavior of the send() system call as it interacts with the networking layer of the local system.

According to the POSIX standard, and the send() documentation I've read, the length of the message to be sent is specified by the length argument. Note that: send() sends one message, of length length. Further:

If space is not available at the sending socket to hold the message to be transmitted, and the socket file descriptor does not have O_NONBLOCK set, send() shall block until space is available. If space is not available at the sending socket to hold the message to be transmitted, and the socket file descriptor does have O_NONBLOCK set, send() shall fail.

I don't see any possibility in this definition for send() to ever return any value other than -1 (which means no data is queued in the kernel to be transmitted) or length, which means the entire message is queued in the kernel to be transmitted. I.e., it seems to me that send() must be atomic with respect to locally queuing the message for delivery in the kernel.

1. If there is enough room in the socket queue in the kernel for the entire message and no signal occurs (normal case), it's copied and returns length.
2. If a signal occurs during send(), then it must return -1. Obviously we cannot have queued part of the message in this case, since we don't know how much was sent. So nothing can be sent in this situation.
3. If there is not enough room in the socket queue in the kernel for the entire message and the socket is blocking, then according to the above statement send() must block until space becomes available. Then the message will be queued and send() returns length.
4. If there is not enough room in the socket queue in the kernel for the entire message and the socket is non-blocking, then send() must fail (return -1) and errno will be set to EAGAIN or EWOULDBLOCK. Again, since we return -1 it's clear that in this situation no part of the message can be queued.

Am I missing something? Is it possible for send() to return a value which is >=0 && <length? In what situation? What about non-POSIX/UNIX systems? Is the Windows send() implementation conforming with this?

Answer 1

Your point 2 is over-simplified. The normal condition under which send returns a value greater than zero but less than length (note that, as others have said, it can never return zero except possibly when the length argument is zero) is when the message is sufficiently long to cause blocking, and an interrupting signal arrives after some content has already been sent. In this case, send cannot fail with EINTR (because this would prevent the application from knowing it had already successfully sent some data) and it cannot re-block (since the signal is interrupting, and the whole point of that is to get out of blocking), so it has to return the number of bytes already sent, which is less than the total length requested.

Answer 2

1. According to the Posix specification and all the man 2 send pages I have ever seen in 30 years, yes, send() can return any value > 0 and <= length. Note that it cannot return zero.

2. According to a discussion a few years ago on news:comp.protocols.tcp-ip where all the TCP implementors are, a blocking send() won't actually return until it has transferred all the data to the socket send buffer: in other words, the return value is either -1 or length. It was agreed that this was true of all known implementations, and also true of write(), writev(), sendmsg(), writev(),

Answer 3

I know how the thing works on Linux, with the GNU C Library. Point 4 of your question reads differently in this case. If you set the flag O_NONBLOCK for the file descriptor, and if it is not possible to queue the entire message in the kernel atomically, send() returns the number of bytes actually sent (it can be between 1 and length), and errno is set to EWOULDBLOCK.

(With a file descriptor working in the blocking mode, send() would block.)

Answer 4

It is possible for send() to return a value >= 0 && < length. This could happen if the send buffer has less room than the length of the message upon a call to send(). Similarly, if the current receiver window size known to the sender is smaller than the length of the message, only part of the message may be sent. Anecdotally, I've seen this happen on Linux through the a localhost connection when the receiving process was slow to unload the data it was receiving from its receive buffer.

My sense is that one's actual experience will vary a good bit by implementation. From this Microsoft link, it's clear that a non-error return value less than the length can occur.

It is also possible to get a return value of zero (again, at least with some implementations) if a zero-length message is sent.

This answer is based on my experience, as well as drawing upon this SO answer particularly.

Edit: From this answer and its comments, evidently an EINTR failure may only result if the interruption comes before any data is sent, which would be another possible way to get such a return value.

Answer 5

On a 64-bit Linux system:

sendto(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4294967296, 0, NULL, 0) = 2147479552

So, even trying to send lowy 4GB, Linux chickens out and sends less than 2GB. So, if you think that you'll ask it to send 1TB and it patiently will sit there, keep wishing.

Similarly, on an embedded system with just a few KBs free, don't think that it'll fail or will wait for something - it'll send as much as it can, and tell you how much that was, letting you to retry with the rest (or do something else in the meantime).

Everyone agrees that in case of EINTR, there can be a short send. But EINTR can happen at any time, so there can always be a short send.

And finally, POSIX says that the number of bytes sent is returned, period. And whole Unix and POSIX which formalizes it is built on the concept of short read/writes, which allows implementations of POSIX systems to scale from the tiniest embedded to supercomputers with proverbial "bigdata". So, no need to try to read between the lines and find indulgences to a particular adhoc implementation you have on your hands. There're many more implementations out there, and as long as you follow the word of the standard, your app will be portable among them.

Answer 6

To clarify a little, where it says:

shall block until space is available.

there are several ways to wake up from that block/sleep:

• Enough space becomes available.
• A signal interrupts the current blocking operation.
• SO_SNDTIMEO is set for the socket and the timeout expires.
• Other, e.g. the socket is closed in another thread.

So things end up thus:

1. If there is enough room in the socket queue in the kernel for the entire message and no signal occurs (normal case), it's copied and returns length.
2. If a signal occurs during send(), then it must return -1. Obviously we cannot have queued part of the message in this case, since we don't know how much was sent. So nothing can be sent in this situation.
3. If there is not enough room in the socket queue in the kernel for the entire message and the socket is blocking, then according to the above statement send() must block until space becomes available. Then the message will be queued and send() returns length. Then send() can be interrupted by a signal, the send timeout can elapse,... causing a short send/partial write. Reasonable implementations will return -1 and set errno to an adequate value if nothing was copied to the send buffer.
4. If there is not enough room in the socket queue in the kernel for the entire message and the socket is non-blocking, then send() must fail (return -1) and errno will be set to EAGAIN or EWOULDBLOCK. Again, since we return -1 it's clear that in this situation no part of the message can be queued.