Alternatives of BinaryFormatter in C#?

Question

I have a file in which I am writing the content using the below C# code.

ConcurrentDictionary<string, DateTime> _jobsAck;
public void SaveToDisk()
{
    var binaryFormatter = new BinaryFormatter();
    using (var stream = File.Open(BINARY_FILENAME, FileMode.OpenOrCreate))
    {
        binaryFormatter.Serialize(stream, _jobsAck);
    }
}

I am reading this file and deserializing using the below C# code.

public void LoadFromDisk()
{
    if (!File.Exists(BINARY_FILENAME)) return;

    var binaryFormatter = new BinaryFormatter();
    using (var stream = File.Open(BINARY_FILENAME, FileMode.Open, FileAccess.Read))
    {
        var deserializedStream = binaryFormatter.Deserialize(stream);
        _jobsAck = deserializedStream as ConcurrentDictionary<string, DateTime>;
        if (_jobsAck == null)
        {
            _jobsAck = new ConcurrentDictionary<string, DateTime>();
            if (!(deserializedStream is Dictionary<string, DateTime> ackDict)) return;
            foreach (var pair in ackDict)
            {
                _jobsAck.TryAdd(pair.Key, pair.Value);
            }
        }
    }
}

We have been asked not to use the BinaryFormatter because it has some security-related issues. So Is there any alternative way which could read/write in binary format?

.Net framework version: 4.7.2

Answer 1

I created a binary serialization alternative, which is fully compatible with the IFormatter infrastructure, including the ISerializable, IDeserializationCallback, IObjectReference interfaces, the (de)serialization event methods, surrogate selectors and binders.

Security note: Please read the security notes at the Remarks section of the BinarySerializationFormatter class. Even if SafeMode is enabled (which eliminates a lot of security issues that BinaryFormatter suffers from), there might be some security concerns if the stream contains non-natively supported types that are resolved by name on deserialization. To address these issues further restrictions will be introduced for SafeMode in the upcoming version. To be completely safe with the current version see the update below.

You can find the NuGet package here and an online demo here, which also demonstrates how compact the result can be compared to BinaryFormatter.

---

Update for clarifying the security questions:

Does your package solve the security issue which is with the BinaryFormatter?

TL;DR: Yes, if you use only the natively supported types by the serializer and enable SafeMode on deserialization. Additionally, until the next version is released it's also recommended to you use a safe binder to filter the expected types encoded by assembly qualified name (see an example at the bottom of the answer).

Elaborated answer:

BinaryFormatter is dangerous at multiple levels. Whereas some of these threats can be cured by a reimplementation (eg. we can prevent auto-loading assemblies referred by the serialization stream, we can use pessimistic allocation for arrays, strings and collections and increase the capacity dynamically to prevent OutOfMemoryException attacks, etc.), others come from the IFormatter instrastructure itself. For example, serializable types that can have an invalid state in terms of their fields usually do not validate the deserialized data in their deserialization events or in the special constructor. So it is partly the implementer's responsibility to ensure security completely.

The main reason of BinaryFormatter "cannot be made secure" that it's a polymorphic serializer, meaning, an object field can hold any serializable type - just name it in the serialization stream and it will be resolved. It is a common misunderstanding that it's insecure because it uses a binary format, whereas JSON serialization is safe. No, polymorphic JSON serialization (eg. Newtonsoft's JSON.NET) is also insecure if you allow the type names to be dumped and resolved. That's why System.Text.Json does not support polymorphism automatically, and applying some polymorphism to it can be a pain.

And vice versa, Google's ProtoBuf is safe, even though it has a binary format because it only uses a few primitive types that are not resolved by name but from a closed set of identifiers. The most complex thing you can encode is a list of key-value pairs. In return, it's really hard to serialize a nested object graph with ProtoBuf.

BinarySerializationFormatter attempts to minimize the risks by supporting a lot of types (including collections) natively. These types are encoded without any assembly identity so it ensures both safety and a very compact payload if you don't use any custom types. As long as you use only these types and enable SafeMode on deserialization you are completely safe.

If you are using any custom type (even if it's just a simple enum) the assembly qualified name of the type must be stored, which can be manipulated. The SafeMode in the current version only ensures that the type can be resolved from the already loaded assemblies only, so it refuses to load new assemblies during the deserialization. But this can be safe only if the already loaded assemblies cannot be exploited by any security hole. For example, if you target .NET Framework, the TempFileCollection class can be exploited to delete files (this specific attack is now protected by a special handling in SafeMode). Therefore in the upcoming versions in SafeMode you must explicitly declare the types that are expected to be resolved by name. Until then you can use one of the binders to be even more safe. Find an example below.

I have already serialized the data using binaryformatter and written to the file. now I want to use some other alternative which can read this file and deserialize the file content.

The format of the streams are not compatible. So BinarySerializationFormatter can only assure that if you were able to serialize your objects with BinaryFormatter, then it will work also with my serializer but the binary stream will be different (in fact, much more compact).

However, if you really need to use the stream serialized by BinaryFormatter, a small step towards security can be using a serialization binder (eg. this one), which can be used also with BinaryFormatter. It can ensure that only the expected types are allowed to be deserialized, and if its SafeMode is true, then unexpected type names will not be resolved even if the serialization stream contains a manipulated assembly identity with a potentially harmful module initializer or a type with malicious constructor, etc.

Answer 2

If your customers already have data saved with binaryFormatter you need to keep it for reading files, regardless of its security issues, until you have migrated all, or most, of your customers to some new format. There is to my knowledge no publish specification of the format for BinaryFormatter, nor any other compatible libraries. And even if there where, I'm not sure it could solve the security problems, since the problems are inherent to the format itself.

So the first step should be to create a new format, using some well designed serialization library. I mostly use json and protobuf (.net), but there are plenty of good alternatives, see https://softwarerecs.stackexchange.com/ if you want recommendations. Just about anything should be better than BinaryFormatter.

You should then update your application so that it can no longer save files using binaryFormtter, only in your new format. Depending on your exact use case you might be able to convert saved data as soon as the new version is installed, in other cases you might only be able to do so when a user explicitly saves a file.

Once your updated application with support for the new format has been out for a while you can start thinking about removing support for BinaryFormatter. Users of older versions might be forced to update to an intermediate version and convert their files. Or you might publish a separate tool that only does conversions between the old format and the new format. You could also add a security warning when opening a file in the old format, to at least warn the user of the risk.

The main point here is that the sooner you introduce a new format, the sooner you can drop support for the old format. The length of this process will largely depend on your support commitments to customers, and willingness to make breaking changes.