2

Client:

import socket
s = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
msg = b"X"
for i in range(1500):
    s.sendto(msg,("<IP>",<PORT>))

Server:

import socket
s = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
s.bind(("",>PORT>))
counter = 0
for i in range(1500):
    s.recv(1)
    counter += 1

I have two machines - the first one with Windows7 and the second one with Ubuntu 16.04.

Now the problem:

If I try to send 1500 UDP-packets (for example) from the client to the server, then:

  • Windows7 is Client and Ubuntu16.04 is server: server only receives between 200 and 280 packets

  • Ubuntu16.04 is Client and Windows7 is server: server receives all 1500 packets

My first question:

What is the reason for this? Are there any limitations on the OS?

Second question:

Is it possible to optimize the sockets in Python?

I know that it will be possible, that UDP-packages can get lost - but up to 4/5 of all packets?

edit: Why this kind of question? Imagine I have a big sensor-network... and one server. Each sensor-node should send his information to the server. The program on the server can only be programmed in an asynchronious way - the server is only able to read the data out of the socket at a specific time. Now I want to calculate how many sensor-nodes can send data via UDP-packets to the server during the period of time where the server is not able to read out his buffer. With the information how many different UDP-packets can be stored in the buffer, I can calculate how many sensor-nodes I can use...

GenXGer
  • 67
  • 8
  • Correct me if i'm wrong, but you're only receiving 1 byte per 1500 iterations? – Torxed Jul 09 '18 at 12:34
  • My first guess would be the windows firewall settings, have you checked if the ports you're trying to use are open? – meissner_ Jul 09 '18 at 12:41
  • @Torxed: yes.. but the messages should be 200-250 bytes long. Now I have tried some other message-lengths, and there are less packets received. How can i figure out how many different packets which a specific size could be stored in the buffer? :/ And why there are some differences on the OS? – GenXGer Jul 09 '18 at 12:57
  • @meissner: there was no problem on the windows-machine with receiving the packets? But I tried to bind the sending-socket on a specific port and opened the port in the windows-firewall - there was the same result, only 1 of 5 packets received on the Ubuntu-machine... I have checked this with wireshark, and wireshark tells me that the windows-machine has send 1500 udp-packets.. – GenXGer Jul 09 '18 at 13:05

1 Answers1

1

Instead of writing a cluttered comment trail, here's a few cents to the problem. As documented by redhat the default values for the different OS:es in this writing moment is:

  • Linux: 131071
  • Windows: No known limit
  • Solaris: 262144
  • FreeBSD, Darwin: 262144
  • AIX: 1048576

These values should correspond to the output of:

import socket
s = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
print(s.getsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF))

These numbers represents how many bytes can be held at any given moment in the socket receive buffer. The numbers can be increased at any given time at the cost of RAM being reserved for this buffer (or at least that's what I remember).

On Linux (And some BSD flavors), to increase the buffer you can use sysctl:

sudo sysctl -w net.core.rmem_max=425984
sudo sysctl -w net.core.rmem_default=425984

This sets the buffer to 416KB. You can most likely increase this to a few megabytes if buffering is something you see a lot of.

However, buffers usually indicate a problem because your machine should rarely have much in the buffer at all. It's a mechanism to handle sudden peaks and to serve as a tiny platter for your machine to store work load. If it gets to full, either you have a really slow code that needs to get quicker or you need to offload your server quite a bit. Because if the buffer fills up - no matter how big it is, eventually it will get full again.

Supposedly you can also increase the buffer size from Python via:

s.setsockopt(socket.SOL_SOCKET,socket.SO_RCVBUF, 1024)

However, again, if your OS is capped at a certain roof - that will supersede any value you put in your python program.

tl;dr:

Every OS has limitations based on optimizations/performance reasons. Sockets, file handles (essentially any I/O operation) has them.

It's common, you should find a lot of information on it. All this information above was mostly found via a search on "linux udp recieve buffer".

Also, "windows increase udp buffer size" landed me on this: Change default socket buffer size under Windows

Final note

As you mentioned, the performance, amount etc can vary vastly due to the fact that you're using UDP. It is prone to data loss at benefit of speed. Distance between servers, drivers, NIC's (especially important, some NIC's have a limited hardware buffer that can cause these things) etc all impact the data you'll receive. Windows do a lot of auto-magic as well in these situations, make sure you tune your Linux machine to the same parameters. A UDP packet consists not only of the ammount of data you send.. but all the parameters in the headers before it (in the IP packet, for instance TTL, Fragmentation, ECN, etc.).

For instance, you can tune how much memory your UDP stack can eat under certain loads, to find out your lower threshold (UDP won't bother checking RAM usage), pressure threshold (memory management under load) and the max value UDP sockets can use per socket.

sudo sysctl net.ipv4.udp_mem

Here's a good article on UDP tuning from ESnet:

Beyond this, you're tweaking to your grave. Most likely, your problem can be solved by redesigning your code. Because unless you're actually pushing 1-10GB/s from your network, the kernel should be able to handle it assuming you process the packets fast enough, rather than piling them up in a buffer.

Torxed
  • 22,866
  • 14
  • 82
  • 131