Sending a StringStream of Binary Data from Cereal with ENet

Question

I have been working on wrapping ENet into a set of easy to use functions for a few weeks now and seem to have a bit of an issue.

I have a std::stringstream and am attempting to send the contents to a remote machine using ENet then reconstruct the std::stringstream on the remote machine.

The reason I need to use a std::stringstream is due to the fact that I'm serializing my data with the Cereal Serialization Library which requires a stream.

With Azoth's help he has identified that I need to be using std::istringstream and std::ostringstream. Previously I was only using std::stringstream which was causing an exception.

However now an exception is being thrown within Cereal at portable_binary.hpp line 156:

throw Exception("Failed to read " + std::to_string(size) + " bytes from input stream! Read " + std::to_string(readSize));

Here's what I'm doing:

void Send(ENetHost* Host)
{
    std::ostringstream SData;
    {
        cereal::PortableBinaryOutputArchive Archive(SData);
        Archive(PacketData);
    }

    std::string Out = SData.str();

    ENetPacket* Packet = enet_packet_create(Out.c_str(), Out.size(), ENET_PACKET_FLAG_RELIABLE);
    enet_host_broadcast(Host, 0, Packet);
}

A Cereal Portable Binary Data Archive is constructed to hold a single vector. The std::ostringstream is sent off to the host using ENet.

This part seems to work okay, I can print the information out before and after and it appears to be the same, albeit some weird symbols, but they print out the same on both ends.

Now a std::istringstream is created on the host with the data we received.

NetPacket(enet_uint8 const* Data)
{
    std::istringstream SData(reinterpret_cast<char const*>(Data));
    {
        cereal::PortableBinaryInputArchive Archive(SData);
        Archive(PacketData);
    }
}

At this point I receive the exception at line:

Archive(PacketData)

I have a feeling the data is being changed somehow when it's sent through ENet and/or I'm not pulling the data out of the std::ostringstream correctly and/or not putting the data back into the std::istringstream correctly.

Thank you very much for your time I greatly appreciate it.

For those finding this by Google, note that stringstream does not work (gives a similar error) with the normal binary archive but ostringstream does, for whatever reason. — Andrew, Jan 28 '17 at 07:14

score 3 · Accepted Answer · answered Apr 05 '14 at 05:57

3

Disclaimer: I'm not familiar with enet.

You are getting this error because you aren't constructing the std::stringstream properly upon receiving the packet. A send/receive pair should look something like:

my_send_function()
{
  std::ostringstream os;
  {    
    cereal::PortableBinaryOutputArchive ar(os);
    ar( whatever_needs_to_be_serialized );
  } // the binary archives will flush their output 
    // immediately, but it's better to
    // use cereal archives in an RAII matter all the time

  std::string data = os.str();

  create_packet(data.c_str(), data.size());
  // send out
}

And then on the receiving end, something like this:

my_receive_function( uint8_t const * data ) // data came from some packet
{
  MyDataType d;

  std::istringstream is(reinterpet_cast<char const *>(data));
  // this is safe to do since we generated the data using c_str(), which added
  // a null terminator to the data
  {
    cereal::PortableBinaryInputArchive ar(is);
    ar( d );
  }
}

The basic idea here: use cereal and some ostringstream to generate a string (which is really just an array of bytes), send those raw bytes over the network, pull them into an istringstream, and then have cereal parse that.

answered Apr 05 '14 at 05:57

Azoth

1,652
16
24

Thank you very much. It's throwing an exception within Cereal now saying "Failed to read " + std::to_string(size) + " bytes from input stream! Read " + std::to_string(readSize) What is the purpose of using the { } after declaring std::istringstream and std::ostringstream? – KKlouzal Apr 07 '14 at 20:02
The braces are to enclose the cereal archive in its own scope so that when the closing brace gets hit the destructor of the archive gets called, forcing it to flush its contents. This isn't used in binary archives but it's a good habit to have if you ever use an XML or JSON cereal archive. Your other comment is too vague to comment on, you'd need to post some code. – Azoth Apr 08 '14 at 23:06
Okay, I understand now thank you, Binary Archives always immediately flush their content, however, other types do not, so it is a good idea to always be in the habit of using this technique in case another archive type is ever used. Thank you. I also apologize for the last comment being too vague, I am editing the original post now to include the new code. – KKlouzal Apr 09 '14 at 00:54
Hard to say what is wrong - it looks fine at the surface. The data you print out will have strange symbols because it's a binary archive, not a text based (e.g. XML or JSON) archive. You could try printing it out on the receiving end and making sure it matches the input and working on that. Once you get the two to match it should work. – Azoth Apr 10 '14 at 03:21
Thank you again for the reply, it does look the same on both ends that was the first thing I tried, the size says 9 on both ends and the data looks the same. Could this be an issue with Cereal if that's the case I'll open a bug report on Github. – KKlouzal Apr 10 '14 at 06:08
If you do go that route, try and create a minimal compiling example of your code to help with debugging. – Azoth Apr 10 '14 at 23:06
1

I was finally able to get it working, I had to create a std::string using the data and size then pass that string into the std::istringstream your answer got me most of the way though it so I marked it as the one, thank you again! Now I've got to figure out shared pointers: [link](http://stackoverflow.com/questions/22999539/templatized-storing-multiple-different-types-in-stdvector/22999645?noredirect=1) – KKlouzal Apr 10 '14 at 23:11

Sending a StringStream of Binary Data from Cereal with ENet

1 Answers1