What is the best way to read and then update record in a binary file with c#

Question

I'm trying to edit some records in a binary file, but I just can't seem to get the hang of it.

I can read the file, but than I can't find the position where I want the record to edit, so I can replace.

This is my code so far:

public MyModel Put(MyModel exMyModel)
{
        List<MyModel> list = new List<MyModel>();

        try
        {
            IFormatter formatter = new BinaryFormatter();

            using (Stream stream = new FileStream(_exMyModel, FileMode.Open, FileAccess.Read, FileShare.Read))
            {
                while (stream.Position < stream.Length)
                {
                    var obj = (MyModel)formatter.Deserialize(stream);
                    list.Add(obj);
                }
            
                MyModel mymodel = list.FirstOrDefault(i => i.ID == exMyModel.ID);
                mymodel.FirstName = exMyModel.FirstName;
                mymodel.PhoneNumber = exMyModel.PhoneNumber;
                
                // Now I want to update the current record with this new object
                // ... code to update
            }

            return phoneBookEntry;
        }
        catch (Exception ex)
        {
            Console.WriteLine("The error is " + ex.Message);
            return null;
        }
}

I'm really stuck here guys. Any help would be appreciated.

I already checked these answers:

• answer 1
• answer 2

Thank you in advance :)

Answer 1

I would recommend just writing all objects back to the stream. You could perhaps just write the changed object and each after it, but I would not bother.

Start by resetting the stream: stream.Position = 0. You can then write a loop an serialize each object using formatter.Serialize(stream, object)

If this is a coding task I guess you have no choice in the matter. But you should know that BinaryFormatter has various problems. It more or less saves the objects the same way they are stored in memory. This is inefficient, insecure, and changes to the classes may prevent you from deserializing stored objects. The most common serialization method today is json, but there are also binary alternatives like protobuf.net.

Answer 2

How you update the file is going to rely pretty heavily on whether or not your records serialize as fixed length.

Variable-Length Records

Since you're using strings in the record then any change in string length (as serialized bytes) or anything other change that affects the length of the serialized object will make it impossible to do an in-place update of the record.

With that in mind you're going to have to do some extra work.

First, test the objects inside the read loop. Capture current position before you deserialize each object, test the object for equivalence, save the offset when you find the record you're looking for then deserialize the rest of the objects in the stream... or copy the rest of the stream to a MemoryStream instance for later.

Next, set stream.Position and stream.Length equal to the start position of the record you're updating, truncating the file. Serialize the new copy of the record into the stream, then copy the MemoryStream that holds the rest of the records back into the stream... or capture and serialize the rest of the objects.

In other words (untested but showing the general structure):

public MyModel Put(MyModel exMyModel)
{
    try
    {
        IFormatter formatter = new BinaryFormatter();
        using (Stream stream = File.Open(_exMyModel))
        using (var buffer = new MemoryStream())
        {
            long location = -1;
            while (stream.Position < stream.Length)
            {
                var position = stream.Position;
                var obj = (MyModel)formatter.Deserialize(stream);
                if (obj.ID == exMyModel.ID)
                {
                    location = position;
                    stream.CopyTo(buffer);
                    buffer.Position = 0;
                    stream.Position = stream.Length = position;
                }
            }
            formatter.Serialize(stream);
            if (location > 0 && buffer.Length > 0)
            {
                buffer.CopyTo(stream);
            }
        }
        return phoneBookEntry;
    }
    catch (Exception ex)
    {
        Console.WriteLine("The error is " + ex.Message);
        return null;
    }
}

Note that in general a MemoryStream holding the serialized data will be faster and take less memory than deserializing the records and then serializing them again.

Static-Length Records

This is unlikely, but in the case that your record type is annotated in such a way that it always serializes to the same number of bytes then you can skip everything to do with the MemoryStream and truncating the binary file. In this case just read records until you find the right one, rewind the stream to that position (after the read) and write a new copy of the record.

You'll have to examine the classes yourself to see what sort of serialization modifier attributes are on the string properties, and I'd suggest testing this extensively with different string values to ensure that you're actually getting the same data length for all of them. Adding or removing a single byte will screw up the remainder of the records in the file.

Edge Case - Same Length Strings

Since replacing a record with data that's the same length only requires an overwrite, not a rewrite of the file, you might get some use out of testing the record length before grabbing the rest of the file. If you get lucky and the modified record is the same length then just seek back to the right position and write the data in-place. That way if you have a file with a ton of records in it you'll get a much faster update whenever the length is the same.

Changing Format...

You said that this is a coding task so you probably can't take this option, but if you can alter the storage format... let's just say that BinaryFormatter is definitely not your friend. There are much better ways to do it if you have the option. SQLite is my binary format of choice :)

Actually, since this appears to be a coding test you might want to make a point of that. Write the code they asked for, then if you have time write a better format that doesn't rely on BinaryFormatter, or throw SQLite at the problem. Using an ORM like LinqToDB makes SQLite trivial. Explain to them that the file format they're using is inherently unstable and should be replaced with something that is both stable, supported and efficient.