How to use boost::serialization with nested structs and minimal code changes?

Question

Currently we use POD which is stored in nested structs. Example:

#define MaxNum1 100;
#define MaxNum2 50;

struct A
{
    int Value[MaxNum1]; 
    char SomeChar = 'a';
};

struct B
{
    A data[MaxNum2];
    float SomeFloat = 0.1f;
};


int main()
{
    B StructBObject = {};
}

We want to enhance our data structures using std::vector just like this:

struct NewA
{
    std::vector<int> Value; 
    char SomeChar = 'a';
};

struct NewB
{
    std::vector<NewA> data;
    float SomeFloat = 0.1f;
};

int main()
{
    NewB StructNewBObject = {};
}

The only argument against this modification is that NewA and NewB are no POD anymore and this makes reading/writing to a file more complicated.

How is it possible to read/write NewA and NewB to a file using boost::serialization with minimal code changes to NewA and NewB? Minimal code changes are important because we use for example big structs which have up to 7 nested levels.

why do you consider moving from `int Value[100];` to `std::vector` an enhanchment? Dont get me wrong, I am among the biggest fanboys of `std::vector`, but a vector is not a fixed sized array. The exact replacement would rather be a `std::array`, or just staying with the c-array isnt that bad either when the size is fixed — 463035818_is_not_an_ai, Feb 19 '20 at 15:22
also see here for PODness of `std::array` : https://stackoverflow.com/questions/3674247/is-stdarrayt-s-guaranteed-to-be-pod-if-t-is-pod — 463035818_is_not_an_ai, Feb 19 '20 at 15:24
@ idclev 463035818: I consider moving to std::vector as an enhancement because often our c-arrays in the structs are not realy of fixed size. Sometimes we use an extra integer to store the number of the maximal element. — BlueTune, Feb 19 '20 at 15:31
Could you give out an example on how you use `boost::serialization` with the `A` and `B` structures? From my attempts, I seem to only get it to work if I create a templated serialize method, for which it makes no difference if it's an array or a vector. — LWolf, Feb 19 '20 at 21:25

score 4 · Accepted Answer · answered Feb 19 '20 at 22:14

You can serialize using boost serialization¹:

template <typename Ar> void serialize(Ar& ar, A& a, unsigned) {
    ar & a.Value & a.SomeChar;
}
template <typename Ar> void serialize(Ar& ar, B& b, unsigned) {
    ar & b.data & b.SomeFloat;
}

Using these, you will already have the correct behaviour out of the box with both the C-array and std::vector approaches.

If you want to keep using fixed-size trivially-copyable types², you can use something like Boost Container's static_vector: it will keep track of the current size, but the data is statically allocated right inside the structures.

TRIPLE DEMO

Here's a triple demo program with three implementations depending on the IMPL variable.

As you can see the bulk of the code is kept invariant. However, for "best comparison" I've made sure that all the containers are at half capacity (50/25) before serialization.

The main program also deserializes.

Live On Coliru

#include <boost/iostreams/device/back_inserter.hpp>
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/stream.hpp>

#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>

#include <boost/serialization/access.hpp>
#include <boost/serialization/is_bitwise_serializable.hpp>
#include <boost/serialization/binary_object.hpp>

#include <iostream>

#if (IMPL==0) // C arrays
    struct A {
        int Value[100]; 
        char SomeChar = 'a';
    };

    struct B {
        A data[50];
        float SomeFloat = 0.1f;
    };

    template <typename Ar> void serialize(Ar& ar, A& a, unsigned) {
        ar & a.Value & a.SomeChar;
    }
    template <typename Ar> void serialize(Ar& ar, B& b, unsigned) {
        ar & b.data & b.SomeFloat;
    }

#elif (IMPL==1) // std::vector
    #include <boost/serialization/vector.hpp>
    struct A {
        std::vector<int> Value;
        char SomeChar = 'a';
    };

    struct B {
        std::vector<A> data;
        float SomeFloat = 0.1f;
    };

    template <typename Ar> void serialize(Ar& ar, A& a, unsigned) {
        ar & a.Value & a.SomeChar;
    }
    template <typename Ar> void serialize(Ar& ar, B& b, unsigned) {
        ar & b.data & b.SomeFloat;
    }

#elif (IMPL==2) // static_vector
    #include <boost/serialization/vector.hpp>
    #include <boost/container/static_vector.hpp>
    struct A {
        boost::container::static_vector<int, 100> Value; 
        char SomeChar = 'a';
    };

    struct B {
        boost::container::static_vector<A, 50> data; 
        float SomeFloat = 0.1f;
    };

    template <typename Ar> void serialize(Ar& ar, A& a, unsigned) {
        ar & boost::serialization::make_array(a.Value.data(), a.Value.size()) & a.SomeChar;
    }
    template <typename Ar> void serialize(Ar& ar, B& b, unsigned) {
        ar & boost::serialization::make_array(b.data.data(), b.data.size()) & b.SomeFloat;
    }

#endif

namespace bio = boost::iostreams;
static constexpr auto flags = boost::archive::archive_flags::no_header;
using BinaryData = std::vector</*unsigned*/ char>;

int main() {
    char const* impls[] = {"C style arrays", "std::vector", "static_vector"};
    std::cout << "Using " << impls[IMPL] << " implementation: ";
    BinaryData serialized_data;

    {
        B object = {};
#if IMPL>0
        {
            // makes sure all containers half-full
            A element;
            element.Value.resize(50);
            object.data.assign(25, element);
        }
#endif

        bio::stream<bio::back_insert_device<BinaryData>> os { serialized_data };
        boost::archive::binary_oarchive oa(os, flags);

        oa << object;
    }

    std::cout << "Size: " << serialized_data.size() << "\n";

    {
        bio::array_source as { serialized_data.data(), serialized_data.size() };
        bio::stream<bio::array_source> os { as };
        boost::archive::binary_iarchive ia(os, flags);

        B object;
        ia >> object;
    }
}

Printing

Using C style arrays implementation: Size: 20472
Using std::vector implementation: Size: 5256
Using static_vector implementation: Size: 5039

How to use boost::serialization with nested structs and minimal code changes?

1 Answers1

TRIPLE DEMO

Final Thoughts

Linked