Boost serialization : forward compatibility fail with input stream error

Question

Following this question : Boost serialize child class I'm trying to support forward compatibility for my archive generated with boost serialization but i'm having trouble reading a newer archive with older code :

    class A {
    public:
        A() {}
        virtual ~A() = default;

    private:
        friend class boost::serialization::access;
        template <class Archive> void serialize(Archive &ar, const unsigned int version) {
            ar &mAttributeFromA;
        }

        std::string mAttributeFromA = "mAttributeFromA";
    };
    BOOST_CLASS_VERSION(A, 0)

    class B : public A {
    public:
        B() {}

    private:
        friend class boost::serialization::access;
        template <class Archive> void serialize(Archive &ar, const unsigned int version)
        {
            ar &boost::serialization::base_object<A>(*this);
            ar &mAttributeFromB;
            if (version == 1)
                ar &mNewAttribute;
        }

        std::string mAttributeFromB = "mAttributeFromB";
        std::string mNewAttribute = "mNewAttribute";
    };

    BOOST_CLASS_VERSION(B, 1)


    class Manager {
    public:
        boost::ptr_vector<A> mListOfA; // can store A or B
    private:
        friend class boost::serialization::access;

        template <class Archive> void serialize(Archive &ar, const unsigned int /*version*/) { ar &mListOfA; }
    };
    BOOST_CLASS_VERSION(Manager, 0)


    int main() {
        Manager  mgr;
        mgr.mListOfA.push_back(new B);
        mgr.mListOfA.push_back(new B);

        std::ofstream ofs("myFile.txt");
        {
            boost::archive::text_oarchive oa(ofs);
            oa << mgr;
        }

        try {
            Manager  mgr2;
            std::ifstream ifs("myFile.txt");
            boost::archive::text_iarchive ia(ifs);
            ia >> mgr2;
            mgr2.mListOfA.at(0);
        } catch(boost::archive::archive_exception e)
        {
            e.what();
        }
    }
BOOST_CLASS_EXPORT(A)
BOOST_CLASS_EXPORT(B)

this will generate the following archive :

22 serialization::archive 13 0 0 0 0 2 3 1 B 1 1
0 1 0
1 15 mAttributeFromA 15 mAttributeFromB 13 mNewAttribute 3
2
3 15 mAttributeFromA 15 mAttributeFromB 13 mNewAttribute

If i try to reload the archive with the same code , everything works perfectly.

However if i try to load the archive with an older version of the code : (Class version is 0 and mNewAttribute is gone)

class B : public A {
    public:
        B() {}

    private:
        friend class boost::serialization::access;
        template <class Archive> void serialize(Archive &ar, const unsigned int version)
        {
            ar &boost::serialization::base_object<A>(*this);
            ar &mAttributeFromB;
        }

        std::string mAttributeFromB = "mAttributeFromB";
    };

    BOOST_CLASS_VERSION(B, 0)

The deserialization throw me an "input stream error"

On Coliru

How can i deserialize new archive with old code ?

-- Edit -- Strangely, if i add A and B objects inside the manager it's working. But fail with only A or only B ...

Just checked that linked answer and it _does_ properly describe the requirements to serialize derived types through pointer-to-base. — sehe, Jan 28 '16 at 09:19
Yes your answer in the other question give me the solution to be able to serialize the hierarchy and support backward compatibility. My problem now , is about forward compatibility (newer archive with old code) — grunk, Jan 28 '16 at 09:52

sehe · Accepted Answer · 2016-01-28T09:18:05.683

Your types aren't polymorphic. Versioning likely has nothing to do with things.

http://www.boost.org/doc/libs/1_60_0/libs/serialization/doc/serialization.html#derivedpointers

It turns out that the kind of object serialized depends upon whether the base class (base in this case) is polymophic or not. If base is not polymorphic, that is if it has no virtual functions, then an object of the type base will be serialized. Information in any derived classes will be lost. If this is what is desired (it usually isn't) then no other effort is required.

You can verify this easily: the vector only deserializes As:

Live On Coliru

#include <boost/serialization/serialization.hpp>
#include <boost/serialization/version.hpp>

class A {
  public:
    A(){}

  private:
    friend class boost::serialization::access;
    template <class Archive> void serialize(Archive &ar, const unsigned int version) { ar &mAttributeFromA; }

    std::string mAttributeFromA = "mAttributeFromA";
};
BOOST_CLASS_VERSION(A, 0)

class B : public A {
  public:
    B(){}

  private:
    friend class boost::serialization::access;
    template <class Archive> void serialize(Archive &ar, const unsigned int version) {
        ar &mAttributeFromB;
        if (version == 1)
            ar &mNewAttribute;
    }

    std::string mAttributeFromB = "mAttributeFromB";
    std::string mNewAttribute   = "mNewAttribute";
};

BOOST_CLASS_VERSION(B, 1)

#include <boost/ptr_container/serialize_ptr_vector.hpp>

class Manager {
  public:
    boost::ptr_vector<A> mListOfA; // can store A or B
  private:
    friend class boost::serialization::access;

    template <class Archive> void serialize(Archive &ar, const unsigned int /*version*/) { ar &mListOfA; }
};
BOOST_CLASS_VERSION(Manager, 0)

#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <sstream>

int main() {
    using namespace boost;

    std::stringstream ss;

    { 
        archive::text_oarchive oa(ss); 
        Manager mgr;
        mgr.mListOfA.push_back(new A);
        mgr.mListOfA.push_back(new B);

        oa << mgr;
    }

    std::cout << ss.str() << "\n";

    { 
        archive::text_iarchive ia(ss); 
        Manager mgr;

        ia >> mgr;

        std::cout << "Deserialized: " << mgr.mListOfA.size() << "\n";
    }
}

Prints

22 serialization::archive 13 0 0 0 0 2 2 1 0
0 15 mAttributeFromA 2
1 15 mAttributeFromA

Deserialized: 2

Solution:

Make the hierarchy actually polymorphic
Add serialization of base object
Register derived types
???
Profit!

Sample (WIP) https://www.livecoding.tv/sehe/

Live On Coliru

#include <boost/serialization/serialization.hpp>
#include <boost/serialization/base_object.hpp>
#include <boost/serialization/export.hpp>
#include <boost/serialization/version.hpp>

class A {
  public:
    A(){}
    virtual ~A() = default;

  private:
    friend class boost::serialization::access;
    template <class Archive> void serialize(Archive &ar, const unsigned int version) {
        ar &mAttributeFromA; 
    }

    std::string mAttributeFromA = "mAttributeFromA";
};
BOOST_CLASS_VERSION(A, 0)

class B : public A {
  public:
    B(){}

  private:
    friend class boost::serialization::access;
    template <class Archive> void serialize(Archive &ar, const unsigned int version)
    {
        ar &boost::serialization::base_object<A>(*this);
        ar &mAttributeFromB;
        if (version == 1)
            ar &mNewAttribute;
    }

    std::string mAttributeFromB = "mAttributeFromB";
    std::string mNewAttribute   = "mNewAttribute";
};

BOOST_CLASS_VERSION(B, 1)
BOOST_CLASS_EXPORT(A)
BOOST_CLASS_EXPORT(B)

#include <boost/ptr_container/serialize_ptr_vector.hpp>

class Manager {
  public:
    boost::ptr_vector<A> mListOfA; // can store A or B
  private:
    friend class boost::serialization::access;

    template <class Archive> void serialize(Archive &ar, const unsigned int /*version*/) { ar &mListOfA; }
};
BOOST_CLASS_VERSION(Manager, 0)

#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <sstream>

int main() {
    using namespace boost;

    std::stringstream ss;

    { 
        archive::text_oarchive oa(ss); 
        Manager mgr;
        mgr.mListOfA.push_back(new A);
        mgr.mListOfA.push_back(new B);

        oa << mgr;
    }

    std::cout << ss.str() << "\n";

    { 
        archive::text_iarchive ia(ss); 
        Manager mgr;

        ia >> mgr;

        std::cout << "Deserialized: " << mgr.mListOfA.size() << "\n";
    }
}

Prints

22 serialization::archive 13 0 0 0 0 2 2 1 0
0 15 mAttributeFromA 3 1 B 1 1
1
2 15 mAttributeFromA 15 mAttributeFromB 13 mNewAttribute

Deserialized: 2

Thanks, this working for me too , but the problem occur when i try to deserialize the archive in an older code base. In your exemple : 1- serialize , 2 - go back to BOOST_CLASS_VERSION(B, 0) instead of 1 and try to deserialize the previous archive. This is what get me the crash — grunk, Jan 28 '16 at 09:47
Have you actually tried with my code? I didn't check because you said in the question it was working well outside `ptr_vector` **and** I could see the _objective_ errors that I fixed in the answer. — sehe, Jan 28 '16 at 10:00
Your exemple is working indeed. I have exactly the same skeleton with my real classes but more complex datas (maps , vector , etc ...) . At least your exemple exclude a problem with ptr_vector. I'm going to do further investigation. Thanks — grunk, Jan 28 '16 at 10:39
It doesn't exclude a problem with ptr_vector. You can see it's there. I ask because _your_ code sample is not complete/self contained. I'm not eager to spend time seeing why my answer doesn't work if all I can see it does. Perhaps you can create a SSCCE of your own? — sehe, Jan 28 '16 at 10:41
I have updated my question , strictly based on your exemple. If you run it once (to generate the archive) and then change the B class to version 0 (removing the new attribute) you'll see the error. — grunk, Jan 28 '16 at 13:52
@grunk http://coliru.stacked-crooked.com/a/e62b3b0f1caa89f1 - I don't see it. Can you SSCCE it? Or can you confirm this exact thing breaks on your platform? (In that case, state your name, rank, OS and versions :)) (**fixed minor typo**) — sehe, Jan 28 '16 at 16:41
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/101969/discussion-between-grunk-and-sehe). — grunk, Jan 29 '16 at 07:24

sena · Answer 2 · 2017-11-28T08:27:17.130

Standard boost archives (including binary) do not support forward (upward) compatibility.

There is a patch for xml archive (proposed 5 years ago, still not included), which allows partial forward compatibility - skipping unknown fields.

There is an experimental 3rdaprty ptree archive, which also supports adding new fields.

Enabling forward compatibility for binary archive will require significant modifications in it.

Google protocol buffers offer forward compatibility out of the box.

Boost serialization : forward compatibility fail with input stream error

2 Answers2

Solution:

Linked