35

While using the default (blocking) behavior on an UDP socket, in which case will a call to sendto() block? I'm interested essentially in the Linux behavior.

For TCP I understand that congestion control makes the send() call blocking if the sending window is full, but what about UDP? Does it even block sometimes or just let packets getting discarded at lower layers?

Jens
  • 69,818
  • 15
  • 125
  • 179
Jocelyn delalande
  • 5,123
  • 3
  • 30
  • 34

2 Answers2

25

This can happen if you filled up your socket buffer, but it is highly operating system dependent. Since UDP does not provide any guarantee your operating system can decide to do whatever it wants when your socket buffer is full: block or drop. You can try to increase SO_SNDBUF for temporary relief.

This can even depend on the fine tuning of your system, for instance it can also depend on the size of the TX ring in the driver of your network interface. There are a few discussions about this in the iperf mailing list, but you really want to discuss this with the developers of your operating system. Pay special attention to O_NONBLOCK and EAGAIN / EWOULDBLOCK.

MarcH
  • 18,738
  • 1
  • 30
  • 25
  • 2
    Ok, following your link I found linux-specific explanation on [this thread](http://www.mail-archive.com/iperf-users@lists.sourceforge.net/msg00113.html). So short simplified answer : _Linux do block sendto on a full sending buffer_. – Jocelyn delalande Nov 12 '10 at 15:11
  • 4
    OK, but now the question is pushed just one step further: under which conditions will Linux let the socket buffer fill up? (as opposed to dropping packets out of it). This is unfortunately a very difficult question. – MarcH Nov 12 '10 at 16:16
  • @MarcH, how about recv,send when the socket is blocking? I think in this case send and recv will block anyway? – Bionix1441 Feb 05 '18 at 13:08
  • 1
    @MarcH according to DS.'s answer [here](https://stackoverflow.com/a/4476788/9784373). It seems like `SO_SNDBUF` for UDP socket is the upper limit of UDP packets (and so `sendto()` never blocks). Or do you think what he said is valid for UNIX but not Linux? Thanks! – HCSF Jun 09 '19 at 02:22
11

This may be because your operating system is attempting to perform an ARP request in order to get the hardware address of the remote host.

Basically whenever a packet goes out, the header requires the IP address of the remote host and the MAC address of the remote host (or the first gateway to reach it). 192.168.1.34 and AB:32:24:64:F3:21.

Your "block" behavior could be that ARP is working.

I've heard in older versions of Windows (2k I think), that the 1st packet would sometimes get discarded if the request is taking too long and you're sending out too much data. A service pack probably fixed that since then.

PypeBros
  • 2,607
  • 24
  • 37
mj_
  • 6,297
  • 7
  • 40
  • 80
  • Ok, good answering element, interesting ; but what I have interest in is mostly the blocking (or absence of blocking) related to congestion/full buffer. – Jocelyn delalande Nov 12 '10 at 14:27
  • ARP is only used for locating hosts on the same subnet - usually the router. And the results are cached for subsequent sends. – selbie Sep 17 '11 at 01:29
  • 3
    When an IP packet goes out it requires the IP address of the remote host, and the MAC address of the **next hop**... – Aviad P. Aug 13 '12 at 03:30
  • Does anyone know if using ioctlsocket()/fcntl() with FIONBIO/O_NONBLOCK would keep the send socket from blocking due to ARP lookup? – Ph0t0n Dec 10 '16 at 02:26
  • @Ph0t0n - In my experience (with Windows), if you try to do a send() when the ARP entry has timed out, the first IP packet gets dropped and an ARP request is sent instead. If you were using TCP, that protocol's error recovery will kick in and save you. If you were using UDP, your packet is just lost. This is completely orthogonal to whether send() blocks you. For TCP, I could see where it might if you are sending a large amount of data in one send(). See my question about UDP and ARP [here](http://stackoverflow.com/questions/15440592/arp-timeouts-why-fixed-periodic) for more information. – T.E.D. May 16 '17 at 14:19
  • ...but if you are using UDP (as this question states) ARP will not block you under any implementation I've seen. Your packet just gets thrown on the floor. – T.E.D. May 16 '17 at 14:24
  • @T.E.D. - I have definitely run into a case where ARP lookup caused blocks in sending. I remember once on a Windows 7 computer I had a loop that would send a UDP packet to 10 different IP addresses via 10 sockets. If all were valid IP addresses with valid destinations, everything worked great, but as soon as you changed one to a valid IP that specified a host that didn't exist, the send for that socket would block causing dropped packets all over the place for all sockets. I did a lot of research, and concluded that it was because of ARP lookup, but it's possible my conclusion was incorrect. – Ph0t0n May 28 '17 at 14:30
  • I can confirm it's blocking. I use sendto with a non blocking socket (ioctlsocket-FIONBIO) and when I have offline devices, that were not available since machine boot, it takes more than a send to end the sendto call... Very annoying, I have 800 devices to control with UDP packets and don't want to create 800 threads... But I can't find a solution. Maybe just a thread that check online devices... Any idea how to solve this issue ? – matthieu Jan 28 '20 at 13:20