1

I'm trying to figure out how to work around a problem in Windows. I'm using C# (net5.0), but if you know the answer in C or C++ that shouldn't be a real problem, because I can call functions in DLLs without an issue.

While testing UDP handling with multicast on Windows 10, I found a problem: when my buffer is too short for the payload coming in, System.Net.Sockets.Socket.ReceiveMessageFrom fills the buffer to its maximum capacity, doesn't set the SocketOptions.Fragmented or SocketOptions.Multicast flags (and setting either before the call is made leads to a "not supported" exception), and the IPPacketInformation.Address field is null. ReceiveMessageFrom's return value is the size of my buffer, not the size of the packet. I cannot seem to get the necessary allocation size (even with SocketOptions.Peek) no matter what I do.

When my buffer is long enough for the payload coming in, (System.Net.Sockets.Socket)s.ReceiveMessageFrom fills the buffer, sets SocketOptions.Multicast, and sets the IPPacketInformation.Address field to the multicast group IP that it received from. The return value in that case is the amount of data actually received, when my buffer is larger than the data received.

On Linux, I can set SocketOptions.Fragmented and it will work correctly: the too-small buffer is filled, but the return value of ReceiveMessageFrom is set to the actual size of the incoming data. Combined with SocketOptions.Peek, this allows me to allocate a new buffer large enough to hold it, and retrieve the full data. (This is very much akin to calling e.g. Windows Registry functions with a 0-length buffer, and being told how big of a buffer you'll actually need.)

The alternative to Linux's way would be to try to allocate a buffer as large as the interface will allow, but System.Net.NetworkInformation.IPInterfaceProperties doesn't have a .Mtu member, while System.Net.NetworkInformation.IPv6InterfaceProperties does. I can't figure out how to get the maximum size of a frame, which is necessary because some drivers support a feature called "jumbo frames" that can be upwards of 64kb.)

To whit: My multicastsender program sends packets that are 26 bytes long, containing the entire uppercase US alphabet from A to Z.

My multicastreceiver program is where I have been making the changes. When I set my receive buffer in that program to less than 26 bytes long, I get the problematic behavior. When I set it to 26 bytes or more long, I get the correct behavior.

This question is not about OS-level or Winsock-level buffers. I will tune those separately, if they need to be. I am specifically trying to ensure that the data that I retrieve with ReceiveMessageFrom is not truncated. (When the OS level doesn't have enough buffer space for the data to be queued, it simply drops the entire packet. It does not write a partial packet to the queue. My application is receiving partial data from the call to ReceiveMessageFrom, and it's not indicating that there is anything that was truncated. I need to figure out how to work around this.)

I am not okay with losing packet space by encoding the size of the data in the area reserved for the data itself, as that will take at least 2 bytes, and I already need to squeeze a lot in here. The UDP header already has a Length field, and that field contains what I need, but I have no access to it.

Thanks for your help!

sjcaged
  • 649
  • 6
  • 25
  • Does this answer your question? [How can I set the buffer size for the underneath Socket UDP? C#](https://stackoverflow.com/questions/2408212/how-can-i-set-the-buffer-size-for-the-underneath-socket-udp-c-sharp) –  Jan 06 '21 at 07:15
  • No, because I'm looking for what is essentially the Length field of the UDP header, and I don't see any way to access it. The answer on that question is "try different values". I'm not going to be at all the computers running this code, and I'm looking for something general that will tell me how big my application-internal buffer needs to be to hold the data from the packet. I'm not looking to change Winsock's buffer size. (Thanks for looking, though!) – sjcaged Jan 06 '21 at 07:20
  • TCP/UDP buffer sizes are always application dependent so _"try different values"_ is the norm I'm afraid. e.g. _["Consider increasing the buffer size if you are transferring large files, or you are using a high bandwidth, high latency connection (such as a satellite broadband provider.)"](https://learn.microsoft.com/en-us/dotnet/api/system.net.sockets.socket.receivebuffersize?view=net-5.0#remarks)_ –  Jan 06 '21 at 09:03
  • These are single packets. They're not going to be high-bandwidth streaming. They're going to be about as often as but a little bigger than most DNS lookups. I just need to figure out how to size my application buffer so I don't undetectably truncate the data with ReceiveMessageFrom. I understand and appreciate your concern, but *this is not helping with the issue I am having*. – sjcaged Jan 06 '21 at 09:15
  • Why not just use a 64KB buffer and be done with it? – selbie Jan 06 '21 at 09:31
  • @selbie Because wasting resources just because they're plentiful is the reason why new computers slow down to the speed of older computers; I'm from the days when 4MB of RAM was huge, and I don't want my computer with 16GB of RAM to slow down to that level. And, because the rest of the Windows APIs allow me to figure out exactly how much more buffer space I need to allocate before they can complete successfully. It looks like this is an oversight on the part of the Winsock programmers, but imagine if Microsoft's DNS server allocated 64KB for every DNS query packet that it expected. – sjcaged Jan 06 '21 at 14:56
  • 1
    You wouldn't need to allocate 64KB per packet. You'd need to allocate 64KB per thread or per socket. Are you doing overlapped I/O ? – selbie Jan 06 '21 at 18:01
  • You are demonstrating a fundamental misunderstanding of how the API works (including c/c++) and your comments are coming off as _"[a `rant` in disguise](https://stackoverflow.com/help/dont-ask)"_. By the way, setting the C# property [`Socket.ReceiveBufferSize`](https://referencesource.microsoft.com/#System/net/System/Net/Sockets/Socket.cs,e767a79281364c76,references) leads to the **native** WinSock function [`setsockopt`](https://learn.microsoft.com/en-us/windows/win32/api/winsock/nf-winsock-setsockopt) being called with an `SO_RCVBUF` argument –  Jan 07 '21 at 02:50
  • ... _["`SO_RCVBUF` - Specifies the total `per-socket` buffer space reserved for receives."](https://learn.microsoft.com/en-us/windows/win32/api/winsock/nf-winsock-setsockopt)_ –  Jan 07 '21 at 02:54
  • https://github.com/kyanha/MulticastTesting is the repo. You are fundamentally misunderstanding my issue: I am bringing the data from the socket buffer into my program. I need to know the size of the byte[] to allocate for it, in my code. There is no need to set a socket option, it's already large enough to handle at least one complete packet. In MulticastReceiver.cs there's an allocation of byte[12], with MulticastSender sending 26 bytes. I cannot find the value 26 anywhere that I have tried, in MulticastReceiver.cs. BSD sockets gives it with Peek|Truncate, Windows doesn't. – sjcaged Jan 07 '21 at 04:38
  • The same code does give me 26 when I execute it on Linux, with SocketOptions.Peek|SocketOptions.Fragmented. It is Windows that doesn't behave that way, so I need to find some other way to get the size of the data in the incoming packet. – sjcaged Jan 07 '21 at 04:42
  • The fact that C# calls the enum for that argument `SocketOptions` is an example of an incorrect name for the purpose. But, that's what we have, and there's nothing I can do to change it. – sjcaged Jan 07 '21 at 04:44
  • On Windows, SocketOption.Peek does the correct thing (it leaves the packet in the socket buffer for the next ReceiveMessageFrom), but it throws with an exception indicating an unsupported operation as soon as SocketOption.Fragmented is set. The BSD sockets API says that Fragmented is supposed to cause the return value of recvfrom() to be the actual length of the data, even if the initially-provided byte[] wasn't large enough. But this is apparently not supported in Winsock? How else do I get the length of the data in the packet? Do I have to make a raw socket to get raw packets to parse? – sjcaged Jan 07 '21 at 04:53
  • If you know its 26 then set your buffer to 26 and be done with it. Case closed –  Jan 08 '21 at 00:33
  • This is the earliest point of my examination of multicast, with static data content. My ultimate direction is to have variable-length packets, and I need to know how large they are. I'd prefer not to do triple-buffering in the stack (Winsock receive packet buffer, ReceiveMessageFrom() to a 64k application buffer, then allocate a new array and copy the data from the application buffer to the correctly-sized array). Winsock's operation already needs to do one variable-length copy. I'm trying to avoid needing to do the second variable-length copy. – sjcaged Jan 08 '21 at 03:43
  • This is a case of premature optimization. You're trying to jump through lots of hoops (some system specific) to keep from using a buffer of at most 64K. Unless you have a measurable impact on performance, it's not worth it. – dbush Jan 08 '21 at 04:30
  • This is a part of my code which, according to the Winsock API, apparently can't adhere to the rest of the Windows API pattern of "get necessary size, allocate necessary size, get actual data". This increases the knowledge requirement for long-term maintainers. Plus, it works properly on dotnet-5.0 for Linux, which means that the aforementioned pattern works on Linux when no other Linux system API follows it, and also necessitates #ifdef'ed platform-specific code.If it can't be done, answer this question with "you can't". – sjcaged Jan 08 '21 at 17:06

0 Answers0