How to implement custom serialization/deserialization for a struct in Julia?

Question

The default implementation of Base.serialize and Base.deserialize do the serialization/deserialization for the whole given object.

What is the correct way of excluding a field from being serialized and still be able to deserialize it correctly?

Here is a simplified code sample:

# The target struct
struct Foo
    x::Int
    y::Union{Int, Void} #we do not want to serialize this field
end

foo1 = Foo(1,2)

# Serialization
write_iob = IOBuffer()
serialize(write_iob, foo1)
seekstart(write_iob)
content = read(write_iob)

# Deserialization
read_iob = IOBuffer(content)
foo2 = deserialize(read_iob)

@show foo1
@show foo2

The output of the above code is:

foo1 = Foo(1, 2)
foo2 = Foo(1, 2)

And the desired result should be:

foo1 = Foo(1, 2)
foo2 = Foo(1, nothing)

Here, I assume that we can define a default value for the missing fields, e.g., nothing for yin the above output.

If you are interested in only serializing part of a type, I'd define this part as a new type instead? It is not appealing to me at least to deserialize an object with incomplete state. I never seen that in any other language, what is the application you have in mind? — juliohm, Feb 27 '18 at 18:53
@juliohm The common use case is a field that caches a value and you do not want to serialize that field. In Java, you can achieve it by annotating a field by `@transient`. Here is a related SO for `@transient` in Java: https://stackoverflow.com/questions/910374/why-does-java-have-transient-fields — Mohammad Dashti, Feb 27 '18 at 20:16

Mohammad Dashti · Accepted Answer · 2020-04-16T15:45:11.803

After digging into the implementation of serialization/deserialization in my current version of Julia (0.6.2), I found a solution. Here is the solution that worked for the example in the question:

# Custom Serialization of a Foo instance
function Base.Serializer.serialize(s::AbstractSerializer, instance::Foo)
    Base.Serializer.writetag(s.io, Base.Serializer.OBJECT_TAG)
    Base.Serializer.serialize(s, Foo)
    Base.Serializer.serialize(s, instance.x)
end

# Custom Deserialization of a Foo instance
function Base.Serializer.deserialize(s::AbstractSerializer, ::Type{Foo})
    x = Base.Serializer.deserialize(s)
    Foo(x,nothing)
end

Now, if you run the test code again:

# The target struct
struct Foo
    x::Int
    y::Union{Int, Void} #we do not want to serialize this field
end

foo1 = Foo(1,2)

# Serialization
write_iob = IOBuffer()
serialize(write_iob, foo1)
seekstart(write_iob)
content = read(write_iob)

# Deserialization
read_iob = IOBuffer(content)
foo2 = deserialize(read_iob)

@show foo1
@show foo2

The test code outputs:

foo1 = Foo(1, 2)
foo2 = Foo(1, nothing)

I should mention that the above solution depends on the current implementation of serialization/deserialization (in Julia 0.6.2) and there's no guarantee about its stability in the future. Hence, I'll still keep an eye on finding a better solution.

Update: The above code does not work after Julia 1.0. Here is the updated code:

using Serialization

# The target struct
struct Foo
    x::Int
    y::Union{Int, Nothing} #we do not want to serialize this field
end

# Custom Serialization of a Foo instance
function Serialization.serialize(s::AbstractSerializer, instance::Foo)
    Serialization.writetag(s.io, Serialization.OBJECT_TAG)
    Serialization.serialize(s, Foo)
    Serialization.serialize(s, instance.x)
end

# Custom Deserialization of a Foo instance
function Serialization.deserialize(s::AbstractSerializer, ::Type{Foo})
    x = Serialization.deserialize(s)
    Foo(x,nothing)
end

foo1 = Foo(1,2)

# Serialization
write_iob = IOBuffer()
serialize(write_iob, foo1)
seekstart(write_iob)
content = read(write_iob)

# Deserialization
read_iob = IOBuffer(content)
foo2 = deserialize(read_iob)

@show foo1
@show foo2

How to implement custom serialization/deserialization for a struct in Julia?

1 Answers1