4

I had two send()s in my C program and looking at wireshark, I realized they were sent out as one TCP/IP message. I am assuming this is some sort of TCP/IP optimization that determined they were small enough that they could be sent out together. However, I am rebuilding an old program from scratch and I am building my tool based on it's TCP/IP traffic: MTU limitations, internal protocol design, etc. So if the old tool sends out two separate messages, I need to send out two separate messages.

So does anyone know what specifically it is doing in the background(besides simple optimization) and if there is a flag or something that needs to be enabled/disabled so that I get a 1 to 1 ratio of C send()s and TCP/IP transmission? For now all I can do to keep them separated is to put a sleep(1) after every send().

Thanks.

JoeManiaci
  • 435
  • 3
  • 15
  • 3
    There are no "messages" in TCP. TCP is a stream protocol. – wildplasser May 14 '14 at 19:49
  • Yes I know, but you still have separate frames that are sent back and forth. – JoeManiaci May 14 '14 at 20:48
  • 1
    That is how the lower network layers happen to work. The steam can be chopped up into arbitrary {chunks,frames,segments} only to be reassembled at the final destination into one continuous stream. That is how your application sees the incoming data, not very different from a pipe, a serial port or a tty. – wildplasser May 14 '14 at 21:07
  • I was driving by the question and had to thought about a flush of the available buffer, and the following answer pointed that out. https://stackoverflow.com/a/7400391/9293869 – JackGrinningCat Apr 12 '18 at 09:43

1 Answers1

9

You can set TCP_NODELAY in setsockopt to disable Nagle's algorithm, to prevent your OS from combining small packets. However, it's important for you to realize that TCP is a stream-oriented protocol, and individual "packets" are not intended to be meaningfully separated. Routers along the way are free to combine or split TCP packets (though this is uncommon, due to the extra processing required), and the receiving OS will not necessarily read exactly one sent packet per recv(). If you want to delineate packets of information in TCP, you'll need to use a header structure to report how many of the following bytes belong to that packet.

Sneftel
  • 40,271
  • 12
  • 71
  • 104
  • As above, I guess I should use "frames" as they are the more accurate term. I definitely am not wanting to separate things out. I, in C, used send() twice, and thus expected to see two dedicated frames to that end. Instead I ended up with one frame. – JoeManiaci May 14 '14 at 20:53
  • 1
    Sorry, TCP just doesn't work that way. If you need this functionality, you'll need to build that **on top** of TCP. – DoxyLover May 14 '14 at 20:55
  • @Soxy, I would disagree considering it probably worked that way before Nagle came along in 86'. It also looks as though TCP_NODELAY(Or anything that looks close) is available in my distro as a socket option. Opensuse 12.3 I believe. – JoeManiaci May 14 '14 at 21:27
  • Nagle's algorithm, and the ability to use TCP_NODELAY to disable it, are available in literally every OS currently available. Look again. – Sneftel May 14 '14 at 21:43
  • And, as @DoxyLover mentioned, TCP doesn't have "frames" either. Two calls to `send()` may mean one block of data returned by `recv()`, or three. There is no foolproof way around this. It's the way TCP was designed to work. – Sneftel May 14 '14 at 21:45
  • Geez, did I absolutely have to use the word datagram to get across what I was trying to accomplish? Obviously not. Hell, technically the wiki used the word chunk the exact number of times as datagram in the section that covers TCP segmentation. – JoeManiaci May 14 '14 at 22:02
  • @JoeManiaci - even if the two TCP sends were to generate two Ethernet frames, there is no guarantee that the receiving system won't present both to one recv. Depending on this will, at best, produce code that will fail "once in a blue moon". Enjoy debugging it. – DoxyLover May 14 '14 at 22:04
  • @Doxy: Well so far it has still worked even with the multiple send()s being optimized, in terms of debugging. I just want to be able to focus on other things when something does break. At the end of the day I would hope to take the TCP_NODELAY out and still have things work. – JoeManiaci May 14 '14 at 22:06
  • Again, what you are going to end up with is code that "works" and will fail sometime when, maybe, a high-priority process delays your process and gives the receiving TCP stack to buffer the data. Or maybe, you upgrade to a new kernel and this behavior changes. This is not what you want. – DoxyLover May 14 '14 at 22:10
  • From everything I have found so far, this will not fail anything, only potentially lead to very high latency and the only thing between me(or a user) and a printer is a switch so I doubt it'll lead to a timeout. But I definitely was worried about an updated kernel causing issues, but like I said, after my testing I am assuming I will be able to allow the optimizations and everything will be fine. I just want to be able to see in wireshark that my behavior lines up 100% against the original tool. Now that I think about it, the old tool never optimizes the way my tool does.......... – JoeManiaci May 14 '14 at 22:24
  • I have (rarely) seen situations where a router didn't honor TCP_NODELAY, resulting in some unexpected slow connections where the one end was sending a single byte and then waiting for a response, but all this stuff should normally be invisible anyway. – user3303729 May 15 '14 at 00:53
  • Exactly, at the very least I am sending 41 Bytes of data just to handle 1 byte of actual traffic. But I probably should have explained this is entirely in a lab environment. I am literally feet from what I am working with. – JoeManiaci May 15 '14 at 14:37