0

I have written and modified a code that sends end receives file using System.Net.Socket. Whenever I send 200MB+ file I get OutOfMemoryException at Dim bytFbyte(intByteSize) As Byte and if the file size is 250-300MB+? I also get OutOfMemoryException at Dim bytFileDataByte() As Byte = File.ReadAllBytes(arrSplit(1)).(Location of the code has comments) How can I correct this. This is my code.

DataSender.vb:

Imports System.IO
Imports System.Net.Sockets
Imports System.Text

Public Class DataSender

    Private strIp As String
    Private intPort As Integer

    Public Sub New(ByRef strIp As String, intPort As Integer, strFileName As String,
                   strFullFileName As String)
        Me.strIp = strIp
        Me.intPort = intPort
    End Sub

    Public Sub SendData(ByVal strArg As String, strMessage As String)
        Dim sckSock = New Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
        sckSock.Connect(strIp, intPort)

        If strArg = "file" Then
            'MsgBox(strMessage)
            Dim arrSplit As String() = strMessage.Split(New Char() {"&"c})
            Dim bytFileNameByte() As Byte = Encoding.UTF8.GetBytes(arrSplit(0))
            Dim bytFileDataByte() As Byte = File.ReadAllBytes(arrSplit(1)) 'System.OutOfMemoryException 250-300MB+ file
            Dim bytFileLen() As Byte = BitConverter.GetBytes(bytFileNameByte.Length)
            Dim intByteSize As Integer = 4 + bytFileNameByte.Length + bytFileDataByte.Length
            Dim bytFbyte(intByteSize) As Byte 'System.OutOfMemoryException 200MB file
            Dim intIdxA As Integer = 0
            Dim intIdxB As Integer = bytFileLen.Length
            Dim intIdxC As Integer = intIdxB + bytFileNameByte.Length

            bytFileLen.CopyTo(bytFbyte, intIdxA)
            bytFileNameByte.CopyTo(bytFbyte, intIdxB)
            bytFileDataByte.CopyTo(bytFbyte, intIdxC)

            If sckSock.Send(bytFbyte) Then
                Dim bytReplyMessage(1024) As Byte
                Dim intBytesReceived As Integer = sckSock.Receive(bytReplyMessage)
                MsgBox(Encoding.UTF8.GetString(bytReplyMessage, 0, intBytesReceived))
            End If

        ElseIf strArg = "command" Then
            Dim test() As Byte = Encoding.UTF8.GetBytes(strMessage)
            sckSock.Send(test)
            Debug.WriteLine("Message sent!")
        End If
        sckSock.Close()
    End Sub
End Class

DataReceiver.vb:

Imports System.IO
Imports System.Net
Imports System.Net.Sockets
Imports System.Text
Imports System.Threading

Public Class DataReceiver

    Private frmBase As Form
    Private thdReceiveListener As Thread
    Private intReceiverPort As Integer
    Private intFlag As Integer = 0
    Private strReceivedPath As String = ""
    Private strReceivedDirPath As String
    Private strMessage As String
    Private rtbDisplay As RichTextBox
    Private Delegate Sub MyDelegate()

    Public Sub New(ByVal frmBase As Form, intReceiverPort As Integer, strReceivedDirPath As String, rtbDisplay As RichTextBox)
        Me.frmBase = frmBase
        Me.intReceiverPort = intReceiverPort
        Me.strReceivedDirPath = strReceivedDirPath
        Me.rtbDisplay = rtbDisplay
    End Sub

    Public Sub StartReceiving()
        thdReceiveListener = New Thread(AddressOf StartListening)
        thdReceiveListener.Start()
    End Sub

    Private Class StateObject
        Public sckWorkSocket As Socket = Nothing
        Public Const intBufferSize As Integer = 1024
        Public bfrBuffer As Byte() = New Byte(intBufferSize - 1) {}
    End Class

    Private Shared mreAllDone As New ManualResetEvent(False)

    Private Sub StartListening()
        Dim bytBytes As Byte() = New [Byte](1023) {}
        Dim ipEnd As New IPEndPoint(IPAddress.Any, intReceiverPort)
        Dim sckListener As New Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)

        Try
            sckListener.Bind(ipEnd)
            sckListener.Listen(100)
            While True
                mreAllDone.Reset()
                sckListener.BeginAccept(New AsyncCallback(AddressOf AcceptCallback), sckListener)
                mreAllDone.WaitOne()
            End While
        Catch ex As Exception
        End Try
    End Sub

    Private Sub AcceptCallback(ar As IAsyncResult)
        mreAllDone.[Set]()
        Dim sckListener As Socket = DirectCast(ar.AsyncState, Socket)
        Dim sckHandler As Socket = sckListener.EndAccept(ar)
        Dim soState As New StateObject()
        soState.sckWorkSocket = sckHandler
        sckHandler.BeginReceive(soState.bfrBuffer, 0, StateObject.intBufferSize, 0, New AsyncCallback(AddressOf ReadCallback), soState)
        intFlag = 0
    End Sub

    Private Sub ReadCallback(ar As IAsyncResult)
        Dim strFileNameLen As Integer = 1
        Dim strContent As [String] = [String].Empty
        Dim soState As StateObject = DirectCast(ar.AsyncState, StateObject)
        Dim sckHandler As Socket = soState.sckWorkSocket
        Dim intBytesRead As Integer = sckHandler.EndReceive(ar)
        Dim strFileName As String = ""

        If intBytesRead > 0 Then
            If intFlag = 0 Then
                strFileNameLen = BitConverter.ToInt32(soState.bfrBuffer, 0)
                Try
                    strFileName = Encoding.UTF8.GetString(soState.bfrBuffer, 4, strFileNameLen)
                    strReceivedPath = Convert.ToString(strReceivedDirPath) & strFileName
                    intFlag += 1
                Catch ex As ArgumentOutOfRangeException
                    Debug.WriteLine("Message received")
                    strMessage = Encoding.UTF8.GetString(soState.bfrBuffer)
                    frmBase.Invoke(New MyDelegate(AddressOf MessageWriter))
                End Try
            End If
            If intFlag >= 1 Then
                Dim wtrRriter As New BinaryWriter(File.Open(strReceivedPath, FileMode.Append))
                If intFlag = 1 Then
                    wtrRriter.Write(soState.bfrBuffer, 4 + strFileNameLen, intBytesRead - (4 + strFileNameLen))
                    Dim bytConfirmMessage() As Byte = Encoding.UTF8.GetBytes("File " & strFileName & " transfered. " & intBytesRead - (4 + strFileNameLen) & " bytes total")
                    sckHandler.Send(bytConfirmMessage)
                    strMessage = "File saved: " + strReceivedPath
                    frmBase.Invoke(New MyDelegate(AddressOf MessageWriter))
                    intFlag += 1
                Else
                    wtrRriter.Write(soState.bfrBuffer, 0, intBytesRead)
                End If
                wtrRriter.Close()
                sckHandler.BeginReceive(soState.bfrBuffer, 0, StateObject.intBufferSize, 0, New AsyncCallback(AddressOf ReadCallback), soState)
            End If
        Else
            frmBase.Invoke(New MyDelegate(AddressOf LabelWriter))
        End If
    End Sub

    Private Sub MessageWriter()
        rtbDisplay.AppendText(strMessage)
        rtbDisplay.AppendText(vbNewLine)
    End Sub

    Private Sub LabelWriter()
        'MsgBox("Data Received")
    End Sub

End Class
conquistador
  • 673
  • 3
  • 11
  • 35
  • Read and send chunks in a loop using a stream, E.g. http://stackoverflow.com/a/2030971/246342 – Alex K. Oct 13 '16 at 15:43
  • @AlexK. How would the receiver know the amount of chunks of the file. Should I add the `file name length` and the `file name` in the first chunk? How can I append the chunks to create the whole file? – conquistador Oct 13 '16 at 16:02
  • 1
    TCP is a stream-based protocol. You are not required to (and in most cases you can not/will not) send nor receive all data in a single Send/Receive call. What I'm about to tell you is not necessary, but to keep full track of the file chunks one option is to implement [**message framing**](http://stackoverflow.com/a/37352525/3740093). Doing so will likely make you lose the streaming ability and you'll have to rewrite pretty much all your code, but on the upside you can keep proper track of your data. – Visual Vincent Oct 13 '16 at 16:22
  • 1
    If you do implement message framing (which, again, is optional - you can still manage without it) keep in mind that you still have to send your data (if there's much of it) in chunks. For my TCP file transfer application I usually send files 8192-65536 bytes at a time (8-64 KB). – Visual Vincent Oct 13 '16 at 16:30
  • Well, this my code will be used for Lan file transfer. I will implement yours and I also forgot about `Socket.SendFile` method – conquistador Oct 13 '16 at 16:39
  • 1
    LAN or not, you're still getting an `OutOfMemoryException` ;). When you're dealing with files (even on the local computer) you're always better of processing them chunk-by-chunk. It might slighlty increase the I/O, but your application will be more stable and might operate faster due to that it doesn't need to allocate as much memory. You also decrease the risk of crashes and `OutOfMemory` exceptions. – Visual Vincent Oct 13 '16 at 19:14
  • 1
    If you need any help with the message framing feel free to ask. – Visual Vincent Oct 13 '16 at 19:17
  • @VisualVincent I tried the `Socket.SendFile` first and modified receiver and it seems working and as the documentation said on `Read.AllBytes` I can only read no more than 2GB. Well, who's gonna 2GB+ play video file in a virtual bulletin board (not fullscreen because of other controls(picturebox,labe..etch)) on a 24-inch display. – conquistador Oct 13 '16 at 23:47
  • Now, the problem is how can i get the filename and extension from the sender to the receiver – conquistador Oct 14 '16 at 00:06
  • Send it as a packet. Serialize a class or just send a normal string. In my TCP application I send a packet with a `FileInfo` header, then I let the endpoint reply with if it wants to receive the file or not. If yes, I start sending the file chunks. – Visual Vincent Oct 14 '16 at 05:27
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/125721/discussion-between-conquistador-and-visual-vincent). – conquistador Oct 14 '16 at 14:49
  • @VisualVincent Is there a way to wait for a reply before executing another code on your `ExtendedTcpClient`? Like `code` -> `wait for reply` -> `code` – conquistador Nov 19 '16 at 04:10
  • Could you be more specific? Why do you need to wait? Can't you just subscribe to the `PacketReceived` event and call your new code from there once you've received a packet? – Visual Vincent Nov 19 '16 at 10:06
  • Here's a quick mockup I made. It's untested and written from the top of my head, but I believe it should work: http://pastebin.com/5LyZtjT3 – Visual Vincent Nov 19 '16 at 10:21
  • What it does is that it creates a temporary event handler for the `PacketReceived` event, checks every time a packet is received if it contains a specific header, and if it does it will execute your code and then remove the temporary handler again. The `...send your data here...` notation is where you send the data to wait for a reply. Put no other code after that unless you don't mind it executing before the reply. – Visual Vincent Nov 19 '16 at 10:27

0 Answers0