5

I am currently sending data over TCP/IP in myserver using something like this 

for str in lst:
  data = str + "\n"
  self._conn.sendall(data)

Now suppose my list has the following two string in it

1-This is statement 1 in list
2-This is statement 2 in list

My client is receiving half of line 2 like this.

   This is statement 1 in list
   This is

I would like to send line1 and then line 2 in the list individually. I understand that TCP/IP works this way in which it will send the entire data that is available to send. I think I could put a delay in after calling self._conn.sendall(data) but i wanted to know what other options I have. I cannot make changes to the receiver of the data and I can only make changes to the sender. So far my only option is adding a delay after each send.

Malt
  • 28,965
  • 9
  • 65
  • 105
James Franco
  • 4,516
  • 10
  • 38
  • 80
  • 1
    The client needs to continue making recv calls until it has everything. I'm not familiar with Python so I can't provide a specific answer. – kicken Jun 15 '16 at 04:07
  • If you want full control over what is sent in each packet, there's also the option of using UDP. – Aya Jun 25 '16 at 14:26

3 Answers3

5

TCP works with streams of data, not individual packets. It's like reading data from a file. The sender puts data in its send buffer, and TCP can decide for itself when to send it. The timing of the arrival at the receiving application depends on when the data was sent and on (often unpredictable) network conditions.

TCP deliveries can be made more predicable if you use the TCP_NODELAY flag in your socket (something like socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1). This would cause TCP to send out data as soon as it arrives in its buffer. But still, there would be no guarantees as to arrival times. This is why any time based solution would break, at least in some cases.

The solution is to divide the data stream into chunks yourself. There are several ways of doing that. Here are a few:

  1. Use fixed length messages - if all messages have a fixed length, the receiver just has to recv() the right number of bytes, process the message, then wait for the same number of bytes.

  2. Send the length of the message before each message. If you want to send the string "blah", encode it as "0004blah" or something similar. The receiver will (always) read the first four bytes (which are 0004) to figure out the number of remaining bytes to read. It will then read the required number of bytes, process the message, and then wait for the next one. It's a robust solution that's also easy to implement.

  3. Use a delimiter. Lines in text files are divided by newline characters (\n). Similarly, you can add a special delimiter byte (or bytes) between messages. For example, you can define that messages always end with a dollar sign ($). Then all the receiver has to do is read from the socket byte by byte until it receives a dollar sign. Of course if you take this approach, you have to make sure that the body of the messages doesn't contain the delimiter character.

Community
  • 1
  • 1
Malt
  • 28,965
  • 9
  • 65
  • 105
0

TCP is based on a stream, not individual messages. So you need to parse the end point of each message yourself. One idea in your case would be to read until you get a newline, then process the line. Note that you might read this:

This is statement 1 in list
This is

Then you need to check to see if you got a newline, process the line, then leave your buffer ready to receive the rest, like this:

This is
John Zwinck
  • 239,568
  • 38
  • 324
  • 436
  • Could you give me an example on how I would do that ? – James Franco Jun 15 '16 at 06:22
  • @JamesFranco: Sure: just read up to 1000 chars into a buffer. Split the buffer on newline (`'\n'`). Parse the first N-1 chunks (these are complete lines) and retain the last chunk. Then continue reading, appending to the buffer and repeating the above steps. – John Zwinck Jun 15 '16 at 12:44
0

TCP has a local buffer that is not sent until it's full. You can force flushing of the local buffer so it's sent after every message, but when the other party receives these packages they get stored in another local buffer and your separation may disappear. TCP is a stream, use it as a stream. You have to use separator characters and when the packets are received you have to separate the messages manually. If you want more control, use UDP packets.

user1238367
  • 505
  • 4
  • 11