Serialization into static data using Boost and IOStreams

Question

I'm trying to take a complex, nested structure and store it directly in a .cpp file as static data. The approach I'd like to take is to take my object (which already supports Boost serialization) and serialize it as a binary archive into a byte array. I could then take that byte array, and walk through it to autogenerate the required .cpp code to hold the binary array. Then, I'd like to deserialize from that byte array back into the object.

So basically, at the end of the day I'd like something like this:

unsigned char* my_obj = { 10, 48, 48, 30, 20 ... }

When I want to use that data, I'd just wrap it in the "byte stream" and pass it into Boost again to deserialize back into my actual object.

My question is: is there some easy way to pass byte arrays around as streams? Boost deals with istreams and ostreams for reading and writing the archives. I don't want to use a stringstream or a filestream, but rather what I suppose may be a custom stream that just acts as a giant byte array for whatever is passed to it.

I feel like there should be a great way to create this custom stream and have it seamlessly work with the Boost serialization... I'm just not really sure where to begin?

Oh yeah? I assumed it wasn't the tool for the job because I didn't really want a string, but maybe you're right! I'll try something out to see if that works... — aardvarkk, Nov 07 '12 at 20:32
Well I am not sure it's the "ideal" tool, but if you want to store the data into a tempory stream that the best I can think of. Or if you can just make string from the tokenised code that would be easy as well. — 111111, Nov 07 '12 at 20:33
If your concern is that `std::ostringstream` will make a copy of the data (and it definitely will), then [this answer](http://stackoverflow.com/a/13059195/636019) has you covered. (Just make sure to open the stream in binary mode.) — ildjarn, Nov 07 '12 at 20:57

Evgeny Panasyuk · Accepted Answer · 2015-04-15T22:58:25.487

unsigned char* my_obj = { 10, 48, 48, 30, 20 ... }

It is better to use:

unsigned char my_obj[] = { 10, 48, 48, 30, 20 ... }

So you will have array, which knows it's size, not just pointer to begin. Also, add const if you don't plan to modify it.

My question is: is there some easy way to pass byte arrays around as streams? Boost deals with istreams and ostreams for reading and writing the archives. I don't want to use a stringstream or a filestream, but rather what I suppose may be a custom stream that just acts as a giant byte array for whatever is passed to it.

Check Boost's array_source and array_sink.

live demo:

#include <boost/archive/binary_iarchive.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/stream.hpp>
#include <iostream>
#include <ostream>
using namespace boost;
using namespace std;

struct Data
{
    double value;

    template<class Archive>
    void serialize(Archive & ar, const unsigned int)
    {
        ar & value;
    }
};

int main()
{
    Data d={3.1415926};
    char buffer[256];
    {
        iostreams::array_sink sink(buffer);
        iostreams::stream<iostreams::array_sink> stream(sink);
        archive::binary_oarchive out_archive(stream);
        out_archive << d;
    }
    Data restored = {0.0};
    {
        iostreams::array_source source(buffer);
        iostreams::stream<iostreams::array_source> stream(source);
        archive::binary_iarchive in_archive(stream);
        in_archive >> restored;
    }
    cout << boolalpha << ( restored.value == d.value ) << endl;
}

Output is:

true

This definitely put me on the right track. I ended up using a very similar technique shown here: http://stackoverflow.com/questions/3015582/direct-boost-serialization-to-char-array/5604782#comment18123762_5604782 In that approach, I didn't need to know the buffer size beforehand, and that's what I needed. Thanks! — aardvarkk, Nov 08 '12 at 15:38
This leaves out a very important case: when the array of bytes you want to store is a member of a class. If you do not a priori know the size of the array of bytes, then you use a unsigned char* member which is allocated in the constructor. However boost cannot serialize pointers to primitive types, it isn't supported (https://stackoverflow.com/questions/19076299/how-do-i-serialize-a-class-containing-pointers-to-primitives/19077087#19077087). std::string could work, if it weren't for the fact that 0x0 (which is perfectly valid for binary data) in a char* is interpreted as the null terminator. — SullX, Nov 28 '15 at 19:50
If you need to serialize an array of bytes that is also a class member, try a std::vector. — SullX, Nov 28 '15 at 19:58

Serialization into static data using Boost and IOStreams

1 Answers1