Serializing a chunked file

Question

I was following the example found here on serializing chunks of a large file. Somehow being new to serialization I am now lost as to what parameters to pass to my Serialize method. I have shelved the approach I was using yesterday because of the OOM exception. Will appreciate your help.

Public Shared Sub ReadAndProcessLargeFile(theFilename As String, ByVal obj As LocalDBObject, Optional whereToStartReading As Long = 0)
Dim bf As New BinaryFormatter() ' Create a binary formatter for this stream.

Using fileStram As New FileStream(theFilename, FileMode.Open, FileAccess.Read)
    Dim buffer As Byte() = New Byte(fileStram.Length - 1) {}
    fileStram.Seek(whereToStartReading, SeekOrigin.Begin)
    Dim bytesRead As Integer = fileStram.Read(buffer, 0, buffer.Length)
    While bytesRead > 0
        bytesRead = fileStram.Read(buffer, 0, buffer.Length - 1)
        'It is here where I am now lost. What parameters do I supply to my Serialize method below 
        bf.Serialize()
    End While
End Using
End Sub

why are you reading the stream if you want to serialize to it? — Gusman, May 13 '16 at 11:18
but you want to serialize or deserialize? you're passing arguments like `whereToStartReading` and the function is named `ReadAndProcessLargeFile`... that does not make sense, you want to store or retrieve an object? — Gusman, May 13 '16 at 11:21
Serialization only for now. I want to serialise contents of a huge file and save in another file (obj) — Simpson Kamonere, May 13 '16 at 11:23
Ehm... serialize huge file? it does less sense... serialization is for serialzation of objects, if you want to duplicate the content of a file then just write it... — Gusman, May 13 '16 at 11:25

score 0 · Answer 1 · answered May 13 '16 at 11:25

0

Hmmm, that code is just not serializing anything, to serialize you should do this:

Public Shared Sub Serialize(theFilename As String, ByVal obj As LocalDBObject)
    Dim bf As New BinaryFormatter() ' Create a binary formatter for this stream.

    Using fileStram As File.Create(theFilename)
        bf.Serialize(fileStram, obj);
    End Using
End Sub

But I suspect this is not what you want, unless you explain a bit better your question is really hard to understand what you want to achieve.

answered May 13 '16 at 11:25

Gusman

14,905
2
34
50

That code you just supplied will give out of memory exception. Tried that already. I wanted one that would break down a large file into chunks then serialize – Simpson Kamonere May 13 '16 at 11:27
it can't be done, serialization does everything internally, so if Serialize throws an exception (which is a bit strange) you can't do it with serialization. Are you sure Serialization is the right tool? you keep saying "serializing a big file", does the LocalDBObject have the content of a file? if that's the case then write the file's content manually, don't serialize it. If you add the LocalDBObject structure to the question would be very helpful to understand what's the problem. – Gusman May 13 '16 at 11:29

Serializing a chunked file

1 Answers1