1

I use booost serialization that way :

Header H(__Magic, SSP_T_REQUEST, 98, 72, 42, Date(), SSP_C_NONE);
Header Z;

std::cout << H << std::endl;
std::cout << std::endl;

char serial_str[4096];

std::memset(serial_str, 0, 4096);

boost::iostreams::basic_array_sink<char> inserter(serial_str, 4096);
boost::iostreams::stream<boost::iostreams::basic_array_sink<char> > s(inserter);
boost::archive::binary_oarchive oa(s);

oa & H;
s.flush();

std::cout << serial_str << std::endl;

boost::iostreams::basic_array_source<char> device(serial_str, 4096);
boost::iostreams::stream<boost::iostreams::basic_array_source<char> > s2(device);
boost::archive::binary_iarchive ia(s2);

ia >> Z;

std::cout << Z << std::endl;

And It works perfectly fine.

Nevertheless, I need to send those packet on a socket. My problem is, how do I know on the other side how many bytes I need to read ? The size of the serialized result is not constant and btw is bigger than sizeof of my struct.

How can I be sure that the data is complete on the other side ? I use circular buffer but with serialisation how to do ?

Thx all

Thibaud Auzou
  • 129
  • 10
  • [This](http://stackoverflow.com/questions/3133621/rule-of-thumb-size-of-boost-archive-in-relation-to-original-serialized-object) and [this](http://stackoverflow.com/questions/3015582/direct-boost-serialization-to-char-array) Q&A seem to give evidence that's it's not really possible to determine the size of a `boost::archive` before the serialization actually took place. But I'm not a `boost` serialization expert at all. – πάντα ῥεῖ May 27 '15 at 23:00

2 Answers2

1

In general it's impossible to predict. It depends (a lot) on the archive format. But with object tracking completely subgraphs might be elided, and with dynamic type information a lot of data could be added.

If you can afford scratch buffers for serialized data, you can serialize to a buffer first, and then send the size (now that you know it) before sending the payload.

There will be overhead for

  • object tracking (serializing through pointers/references)
  • dynamic polymorphism (serializing through (smart) pointer-to-base)
  • versioning (unless you disable it for the types involved)
  • archive header (unless disabled)
  • code conversion (unless disabled)

Here are some answers that give you more information about these tweak points:

If all your data is POD, it's easy to predict the size.

Out of the box

If you share IPC on the same machine, and you're already using circular buffers, consider putting the circular buffer into shared memory.

I have lots of answers (search for managed_shared_memory or managed_mapped_file) with examples of this.

A concrete example, focusing on a lock-free single-producer/single-consumer scenario is here: Shared-memory IPC synchronization (lock-free)

Even if you choose to/need to stream messages (e.g. over the network) you can still employ e.g. Managed External Buffers. Hereby you avoid the need to do any serialization even without requiring all data to be POD. (The trick is that internally, offset_ptr<> is used instead of raw pointers, making all references relative).

Community
  • 1
  • 1
sehe
  • 374,641
  • 47
  • 450
  • 633
-1

Create your own streaming class and override xsputn method.

class counter_streambuf : public std::streambuf {
public:
    using std::streambuf::streambuf;

    size_t size() const { return m_size; }

protected:
    std::streamsize xsputn(const char_type* __s, std::streamsize __n) override 
    { this->m_size += __n; return __n; }

private:
    size_t m_size = 0;
};

Usage:

Header H(__Magic, SSP_T_REQUEST, 98, 72, 42, Date(), SSP_C_NONE);
counter_streambuf csb;
boost::archive::binary_oarchive oa(csb, boost::archive::no_header);
oa & H;
cout<<"Size: "<<csb.size();
AshFTW
  • 141
  • 3
  • 12
  • But that gives you the size _after_ serialisation. Unless you hold back the data for transport (as covered much more thoroughly by sehe four years ago) that is too late. – Lightness Races in Orbit Feb 03 '19 at 17:14