Visual Basic TCP Data Corruption

Question

I have a program that's using TCP for communication between the client and server. Client sends a byte array of a BMP image. The listener reads the bytes in chunks and joins the bytes together to get the image. When the server/listener receives the image, it is corrupted almost halfway through.

When using the localhost to test on my computer the corruption is less likely to happen (greater the byte array size = more corruption), however when over the internet it is corrupted 99% of the time regardless of size. I have also tried splitting the byte array and sending it to the server in chunks, this seems to fix the corruption on localhost but is still the same when over the internet.

Can soomeone tell me what I'm doing wrong here?

The listener code running in a timer:

If Listener.Pending = True Then
Client = Listener.AcceptTcpClient()
Message = ""
Dim ClientNetworkStream As NetworkStream = Client.GetStream
ReDim ReadBytes(-1)
If ClientNetworkStream.CanRead = True Then
    Dim ReadBuffer(65535) As Byte
    Dim NumberOfBytesRead As Integer = 0
    Do
    NumberOfBytesRead = ClientNetworkStream.Read(ReadBuffer, 0, ReadBuffer.Length)
    If Not NumberOfBytesRead = 0 Then
        If Not NumberOfBytesRead = -1 Then
            ReadBytes = AppendByteArrays(ReadBytes, ReadBuffer, NumberOfBytesRead)
            If ConvoOngoing = False Then
            ReDim CurrentConvoBytes(-1)
            ConvoOngoing = True
            CurrentConvoBytes = ReadBytes
        Else
            CurrentConvoBytes = AppendByteArrays(CurrentConvoBytes, ReadBytes)
        End If
    End If
End If
Loop While ClientNetworkStream.DataAvailable
    If ClientNetworkStream.DataAvailable = False Then
        If ConvoOngoing = True Then
            If ReadBytes.Length > 0 Then
                CurrentConvoBytes = AppendByteArrays(CurrentConvoBytes, ReadBytes)
            End If
        ConvoOngoing = False
        PictureBox1.Image = ConvertByteArrayToBMP(CurrentConvoBytes)
        End If
    End If
End If
End If

Here is the sender code:

    Private Sub SendWhole(ByVal BytesToSend() As Byte, ByVal Optional isFile As Boolean = False)
    Client = New TcpClient(TextBox2.Text, TextBox3.Text)
    Dim ClientStream As NetworkStream = Client.GetStream
    ClientStream.Write(BytesToSend, 0, BytesToSend.Length)
    ClientStream.Close()
End Sub

And the code for AppendByteArrays used in the code:

    Public Function AppendByteArrays(ByVal OriginalArray As Byte(), ByVal ArrayToAppend As Byte(), ByVal Optional AmountOfValuesToAdd As Integer = -1) As Byte()
    Dim IndexToStart As Integer = OriginalArray.Length
    Dim Modifier As Integer = 0
    If Not AmountOfValuesToAdd = -1 Then
        Modifier = AmountOfValuesToAdd - 1
    Else
        Modifier = -1
    End If
    If Modifier = -1 Then
        ReDim Preserve OriginalArray((OriginalArray.Length) + (ArrayToAppend.Length - 1))
        For i = 0 To ArrayToAppend.Length - 1
            OriginalArray(IndexToStart + i) = ArrayToAppend(i)
        Next
    Else
        ReDim Preserve OriginalArray((OriginalArray.Length) + Modifier)
        For i = 0 To Modifier
            OriginalArray(IndexToStart + i) = ArrayToAppend(i)
        Next
    End If
    Return OriginalArray
End Function

Edit: Here's the bit where I can't seem to figure out the math right

        If Listener.Pending = True Then
        Client = Listener.AcceptTcpClient()
        Dim ClientNetworkStream As NetworkStream = Client.GetStream
        ReDim ReadBytes(-1)
        If ClientNetworkStream.CanRead = True Then
            Dim ReadBuffer(65535) As Byte
            Dim NumberOfBytesRead As Integer = 0
            Do
                NumberOfBytesRead = ClientNetworkStream.Read(ReadBuffer, 0, ReadBuffer.Length)
                If Not NumberOfBytesRead = 0 Then
                    If Not NumberOfBytesRead = -1 Then
                        If ReadingPacket = False Then
                            If PacketHasLeftOvers = True Then
                                PacketHasLeftOvers = False
                                ReadBuffer = AppendByteArrays(PacketLeftOvers, ReadBuffer)
                                ReDim PacketLeftOvers(-1)
                            End If
                            ReadingPacket = True
                            PacketToReadSize = BitConverter.ToInt32(ReadBuffer, 0)
                            PacketType = ReadBuffer(4)
                            ReDim CurrentConvoBytes(-1)
                            ReadBytes = AppendByteArrays(ReadBytes, ReadBuffer.Skip(5).ToArray, NumberOfBytesRead - 5)
                            CurrentConvoBytes = ReadBytes
                            PacketReadSize = ReadBytes.Length - 1
                            If PacketReadSize = PacketToReadSize Then
                                'Check if already finished reading
                                PictureBox1.Image = ConvertByteArrayToBMP(CurrentConvoBytes)
                                PacketReadSize = 0
                                ReadingPacket = False
                            End If
                        Else
                            Dim PacketScore As Integer = PacketReadSize + NumberOfBytesRead
                            If PacketScore < PacketToReadSize Then
                                ReadBytes = AppendByteArrays(ReadBytes, ReadBuffer, NumberOfBytesRead)
                                CurrentConvoBytes = AppendByteArrays(CurrentConvoBytes, ReadBytes)
                                PacketReadSize += ReadBytes.Length - 1
                            ElseIf PacketScore > PacketToReadSize Then
                                ReadBytes = AppendByteArrays(ReadBytes, ReadBuffer, NumberOfBytesRead - (PacketToReadSize - PacketReadSize))
                                CurrentConvoBytes = AppendByteArrays(CurrentConvoBytes, ReadBytes)
                                PacketHasLeftOvers = True
                                PacketLeftOvers = ReadBuffer.Skip(PacketToReadSize - PacketReadSize).ToArray
                                PictureBox1.Image = ConvertByteArrayToBMP(CurrentConvoBytes)
                                PacketReadSize = 0
                                ReadingPacket = False
                            ElseIf PacketScore = PacketToReadSize Then
                                ReadBytes = AppendByteArrays(ReadBytes, ReadBuffer, NumberOfBytesRead)
                                CurrentConvoBytes = AppendByteArrays(CurrentConvoBytes, ReadBytes)
                                PictureBox1.Image = ConvertByteArrayToBMP(CurrentConvoBytes)
                                PacketReadSize = 0
                                ReadingPacket = False
                            End If
                        End If
                    End If
                Else
                    Throw New Exception
                End If
            Loop While ClientNetworkStream.DataAvailable
        End If
    End If