21

sendfile() copies data between two file descripters within kernel space. Somewhere I saw if you are writing a web server in C in linux you should use send() and recv() instead of using write() and read(). So is the send() use the kernel space as well?

Whatever I use for sending - sendfile() or send() - on the client side I'll be using recv() right?

On the flip side, man page says: "The only difference between send() and write(2) is the presence of flags. With a zero flags argument, send() is equivalent to write(2)."

samsamara
  • 4,630
  • 7
  • 36
  • 66

3 Answers3

41

If fd is a socket file descriptor, then these system calls are identical:

  • send(fd, data, length, 0) is the same as write(fd, data, length)
  • recv(fd, data, length, 0) is the same as read(fd, data, length)

So, unless you need to set a non-zero flags parameter, it makes no difference whether you use send/recv or write/read.

The sendfile system call is an optimization. If you have a socket sockfd and a regular file filefd and you want to copy some file data to the socket (e.g. if you're a web server serving up a file), then you might write it like this:

// Error checking omitted for expository purposes
while(not done)
{
    char buffer[BUFSIZE];
    int n = read(filefd, buffer, BUFSIZE);
    send(sockfd, buffer, n, 0);
}

However, this is inefficient: this involves the kernel copying the file data into userspace (in the read call) and then copying the same data back into kernel space (in the send call).

The sendfile system call lets us skip all of that copying and have the kernel directly read the file data and send it on the socket in one fell swoop:

sendfile(sockfd, filefd, NULL, BUFSIZE);
Adam Rosenfield
  • 390,455
  • 97
  • 512
  • 589
  • so on the other side, the web browser (client) can just read the content. right? – samsamara Nov 04 '12 at 04:42
  • @user601Lwillmywastedtimebe: this has only effect on the efficiency on the server. It is still a TCP/IP connection - this is invisible to the client. – Klaas van Gend Nov 04 '12 at 08:00
3

As you have pointed out, the only difference is the flags. send/recv are for networking, whereas read/write are general I/O functions for any file descriptor. send is only useful vs write when you want to use a flag, since the flags are all network related, it doesn't make sense to call send on a non-network file descriptor (nor am I sure whether it's even valid).

Also you should note:

The in_fd argument must correspond to a file which supports mmap(2)-like operations (i.e., it cannot be a socket).

Which means you can't copy from a socket (you can copy to a socket and prior to 2.6.33 you must copy to a socket).

CrazyCasta
  • 26,917
  • 4
  • 45
  • 72
  • So should I use sendfile() instead of send() if I'm writing a web server? but you said 'send/recv are for networking' not sendfile(). – samsamara Nov 04 '12 at 03:12
  • See Adam's response. The short version of the answer however is, it depends. Basically send/recv are useful because you can use flags. Specifically in a production grade webserver for high volume you should be using non-blocking I/O by using the MSG_DONTWAIT flag (not sure if there's a way other than flags to do this). – CrazyCasta Nov 04 '12 at 03:15
1

send is specified by the POSIX standard, which says:

The send() function is equivalent to sendto() with a null pointer dest_len argument, and to write() if no flags are used.

sendfile is Linux-specific. It tells the kernel to do zero-copy I/O from a file to a socket. (Note that it only works when the source fd is a file and the destination is a socket; for generic Linux-specific zero-copy I/O, read about splice().)

Note that there is rarely any need to use Linux-specific zero-copy I/O. The standard and portable read+write (or send) loop with a small userspace buffer (8K-16K) will generally keep that buffer in L1 cache, making it equivalent to "zero-copy" from the system RAM's point of view.

So unless your profiling shows a difference for your particular application, stick to standard interfaces. Just MHO.

Nemo
  • 70,042
  • 10
  • 116
  • 153
  • Well, you're still going to have 2 resident copies of the same thing in L1 cache (unless the kernel uses some feature I'm unaware of to prevent this) which would push other things out of L1 cache. Also, the copying is likely to take on the order of the number of bytes to copy clock cycles (possible something like number of bytes to copy/4 or /8). – CrazyCasta Nov 04 '12 at 03:17
  • sendfile() is also available on OS X not just Linux. – Paulpjmmchugh May 26 '14 at 00:06
  • postix standard link isn't going anywhere any longer it seems. – luckydonald Nov 02 '18 at 15:50
  • @luckydonald: I tried it once and it failed, then I tried again and it worked... Some kind of infrastructure problem maybe? – Nemo Nov 03 '18 at 18:15
  • Is DMA involves CPU caches or not? In a case of ARM it looks possible to bypass most drive/ethernet data transfer with DMA almost without CPU involved, using bus matrix and special file system holds files in a sequential ordered sectors (especially with NAND/NOR that mapped directly into address space). – Dmitry Ponyatov Dec 05 '22 at 17:15