0

My understanding is that TCP is considered "reliable" because the receiver acknowledges packet receipt and requests a resend if there is any problem. My file transfer program currently sends files in 32767 byte packets, though I have experimented with all sizes. Sending a 10 meg file that requires 340 packets consistently results in three or four packets on the receiver being significantly smaller than what was sent. I always end up with a file that is very slightly different from the original.

As an example, my log records the size of all packets received:

TCP packet received (32767 bytes)
TCP packet received (32767 bytes)
TCP packet received (14600 bytes)
TCP packet received (32767 bytes)

My sending thread reads the file in 32767 byte chunks and calls a sending sub:

MyFile.Read(Buffer, 0, BufferSize)
SendTCPData(Address, Buffer)

My TCP code is very simple:

Shared Sub SendTCPData(Address As String, ByVal Data As Byte())
    Dim Client As New TcpClient(Address, PortNumber)
    Dim Stream As NetworkStream = Client.GetStream()
    Stream.Write(Data, 0, Data.Length)
    Stream.Close()
    Client.Close()
End Sub

Can anyone help? (The post "TCP Client to Server communication" does not deal with with how to handle received packet sizes, which is my question.)

jbbarnes77
  • 119
  • 2
  • 10
  • how are you sending 32KB - 1byte...and why 32767? and why not an even number like 32768? Even there you should consider IP headers to fit let say a fixed size buffer like that. It won't prevent you from having your issue. You never checksum your file during upload? Instead of dumping the file and assume TCP will control the flow without errors you should send an ACK to the client to send the next packet. If you don't want to do it every packet for speed then you should number them to be able to resend the broken packet. – Chillzy Mar 16 '18 at 05:09
  • TCP is byte-oriented. When you give the TCP-layer a 32767-byte chunk it will divide it into segments that are sent in different packets. How do you write the received data to a file? – FormerNcp Mar 16 '18 at 08:54
  • 1
    Possible duplicate of [TCP Client to Server communication](https://stackoverflow.com/questions/35233852/tcp-client-to-server-communication) – Visual Vincent Mar 16 '18 at 11:21
  • 32767 was used in example code I found. It's just the max size of an unsigned 16-bit value, kind of arbitrary. My (simplified) receiving code looks like this. It gets called every time a packet is received: – jbbarnes77 Mar 16 '18 at 18:28
  • Private Sub ReceiveFile(Address As String, ByVal Data As Byte()) If Data.Length = 0 Then Exit Sub 'Some code removed to simplify example Try Dim FS As FileStream = New FileStream(FileName, FileMode.Append, FileAccess.Write) FS.Write(Data, 0, Data.Length) FS.Close() Catch ex As Exception WriteLog("EXCEPTION " + ex.ToString + " in ReceiveFile") End Try End Sub – jbbarnes77 Mar 16 '18 at 18:31
  • While the post @Visual Vincent suggested has some very good code in it, it does not deal with receiving and handling packets that are incomplete, which is my question. – jbbarnes77 Mar 16 '18 at 19:00
  • The packets that arrive to your application will never be incomplete unless you send them incorrectly. If a received packet is incomplete then TCP will re-send it automatically. Your application won't notice a thing. – Visual Vincent Mar 16 '18 at 19:31
  • 1
    The problem with your code is that it assumes that _one send = one receive_, which is not true. TCP is a streamed protocol, which means it has no notion of packets at the application layer. When you call `Read()` it will read everything that it has downloaded _**so far**_, thus that can be less than the size of the actual data (which is the problem you are experiencing). My code takes care of that by keeping track of how much data each "packet" contains, and it doesn't deliver the data to you until it has read a whole packet. For info on how this works see: https://stackoverflow.com/a/37352525 – Visual Vincent Mar 16 '18 at 19:39

1 Answers1

1

TCP doesn't provide a packet interface to applications, it provides a byte stream interface. TCP does not "glue" bytes together into messages -- it's not a message protocol. If you want code that reads and writes messages using TCP, you'll have to actually write it or use someone else's code that does that.

If you know the sender is sending exactly 32,767 bytes, just keep calling TCP receive functions until you get 32,767 bytes. If you don't know exactly how many bytes the sender is going to send or can't identify the end of the data with some kind of marker, then it's impossible to know when you've gotten all of them.

For the future, before you write any networking code, it's worth taking the effort to document the protocol you're going to use. Take a look at some specifications for existing protocols that layer over TCP (such as SMTP, NNTP, FTP, or HTTP) to see what you need to decide on and document.

If you're sending files over TCP, look closely at some standard for sending files over TCP (such as FTP) and either implement that standard or choose a rational subset of it. At a minimum, reading the standard will give you an idea of the types of decisions you need to make to wind up with a protocol that works. Also, it's essential for debugging -- if the program doesn't work, without a standard it's difficult to determine if it's the server or the client that's at fault because there is no reference to compare them to.

David Schwartz
  • 179,497
  • 17
  • 214
  • 278