-1

I am on working a project with Socket TCP/IP ( Server-C# and Client-Python). Streaming video after sometimes,the data of Recv Socket is splitted. My data is buff = 22000 bytes,if it is splitted it will become :

buff = 1460

buff = 20600

I don't know why,i have researched some methods with MTU,Fragmentation,Windows Size,....but not have result Specially,if i setsocketopt the process will appear less.

self.sk.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 1048576)

enter image description here --Image about the data is splitted

enter image description here

This is my Recv Socket.

buff = self.sk.recv(1048576)
print("BUFF RECEIVE  :::  ::::: ---->>>>>      ",len(buff))
if buff == b'' :
  self.sk=None
  buff = None
  return buff

Suggestions: This just only happened to Chrome Browser(It mean,can't Streaming Video if loss data ).But at Firefox,it is not.It seem to blink a moment if loss data but It can continue stream after. enter image description here - Chrome and FireFox

vanloctc
  • 1
  • 4
  • 1
    I have no idea what your title has to do with the body of your question. I don't even understand what your title means. But what you see is perfectly normal - in your user space code you should never treat TCP as a sequence of packets but only as a byte stream. It is up to the OS to split the byte stream into packets and to combine it back to a byte stream in the recipient. A `send` in the sender does not need to exactly match a single `recv` in the recipient. TCP is not a datagram protocol like UDP but a byte stream. – Steffen Ullrich Feb 07 '20 at 07:07
  • Thank you response quickly.About my issue,when I get data from Server,I will use that data to Streaming on Web.But some time,the data is splitted to 1,2,3 segments and I don't know that data is correct?.Example: with buff = 22000 bytes it will streaming OK(Video on Web) but If the buff is splitted so the Web can not stream Video. – vanloctc Feb 07 '20 at 07:25
  • 1
    It is your code that is incorrect, not the Web. Any assumption you may make about the number of bytes you will receive each time is false. The only guarantee is that you will either receeive at least one byte or encounter end of stream or an error. Your title remains a mystery. Please fix it. – user207421 Feb 07 '20 at 07:29
  • @user207421 , I know my title is very hard to understand.But in my case,hard to explain.Thank you for answer,i will consider again. – vanloctc Feb 07 '20 at 09:21
  • It is impossible to understand. It isn't even an English sentence. It also has nothing to do with your question, which is about received chunk sizes, and that in turn is not complicated. 'Asynchronous' is not a verb, and it has nothing to do with chunking whatsoever. Don't create difficulties where none exist. – user207421 Feb 07 '20 at 10:40
  • @user207421. Sorry because I don't know how to express it a clearly way for you .In my case,when the Data is splitted, "Web Streaming" can not stream Video but If I refresh "Web Streaming".It's OK,the Streaming will continue and if it is not enough data again , the Streaming also continue .... can not stream .... I think i can tell you clearly if I have your account(fb,sky,....). – vanloctc Feb 07 '20 at 16:49
  • @user207421. You can see buff = 1460 , 2920 , ...and more 4380,5840,... .This is MTU size,Is that right ?.Is it frag or split at transfer layer?,I don't know, too many things to understand. – vanloctc Feb 07 '20 at 17:15
  • 'Synchronize data' is no improvement. You've been told several times what's really happenong here. – user207421 Feb 07 '20 at 22:13
  • 1
    Are you trying to receive exactly 1,048,576 bytes? If so, just keep calling the `recv` function (passing in the number of bytes you still want) until you've received that many. If you don't know how many bytes you want to read, then you're sunk. if you don't know what you want, you have no way to ask for it and must change your design. – David Schwartz Feb 11 '20 at 04:18
  • Thank @DavidSchwartz ,Assume that my data = 22xxx if i set So_rcvbuff less than so the rate "can't streaming video" higher , and if i set larger than so the rate "can't streaming video" lower.I have set 1,048,576 bytes with so_rcvbuff, it's good better but it still have error. – vanloctc Feb 11 '20 at 04:29
  • 1
    @vanloctc What error are you talking about? – David Schwartz Feb 11 '20 at 11:44

2 Answers2

1

That is just the way TCP works. It is a streaming protocol with no built-in message framing, so it is free to place bytes into packets in any amounts it chooses — all it guarantees is that the bytes will still be in the correct order when they are read() by the receiving socket.

Therefore, if your program requires a certain number of bytes before it can proceed, it is up to your program to do the necessary buffering to assemble those bytes together.

As for why TCP might behave the way you observed —it is likely reacting to network conditions (dropped packets, feedback from the receiving host’s TCP stack, etc), and trying to make transmission as efficient as possible given its current environment. It’s 100% up to the TCP stack how it wants to transmit the data, and different TCP stacks may behave differently, which is fine as long as they follow the rules of the TCP specification.

Jeremy Friesner
  • 70,199
  • 15
  • 131
  • 234
  • ,Thank your answer. As you know, 1 stream will have 22xxx data but as you see, have two streams with **2920 + 19671** and **1460 + 21143** it seem to be MTU size.I can understand that the package is splitted or drop so I think that should have "synchronize" to connect two data together. – vanloctc Feb 09 '20 at 16:12
  • 1
    TCP is not required to behave the way you think it ought to behave; it is only required to behave in ways that are consistent with the TCP spec. – Jeremy Friesner Feb 09 '20 at 21:37
  • More suggestion,this just only happened to Chrome ( it mean,can't streaming Video when loss data ) but not at FireFox.. – vanloctc Feb 11 '20 at 03:19
  • 1
    @vanloctc Data is not being lost. Just call the read function again if you want more data. It's your job to connect the data back together if you have some reason to think it should be connected. – David Schwartz Feb 11 '20 at 04:17
  • @DavidSchwartz,I changed to (https://stackoverflow.com/questions/17667903/python-socket-receive-large-amount-of-data) ,but it is not OK. – vanloctc Feb 11 '20 at 06:01
-1

After a long time,I have found the answer for my issue.

Solution for TCP/IP client socket message boundary problem

**1/**When you send a package from Server to Client with Send(Write).At Client side,the Receive will not get full data in sometimes.It not mean,Send/write at Server not send enough data.Just because this is TCP/IP protocol,Receive is not graduatee and the package will be fragmentation at Client Side ( your code ).

**2/**You can solve this issue by add more pattern at send/write Server Side. For example, send(data) --> send ( Pattern + data) and at Client side,you can use patern to check data.

**3/**Limitations of this method,the package after fragmentation,it can "combine together" or sometime it can't not.For example,your send data = 4000 and at Client side,your receive = 1460 + 2540. This is what I understood with my issue.

vanloctc
  • 1
  • 4
  • You were told all this over a week ago. There is no 'at long last' or 'I have found' about it. And the word isn't 'package', it s 'packet'.i – user207421 Feb 16 '20 at 07:32
  • @user207421,I'm really sorry if my stupid affects you.If you have a link about this issue,please give me and i can learn more from you. – vanloctc Feb 16 '20 at 08:09
  • Link? There is an answer anive and a dozen comments that all tell you the same thing. What *you* need is a link supporting your expected behaviour. – user207421 Feb 17 '20 at 10:56