Serialize and deserialize vector in binary

Question

I am having problems trying to serialise a vector (std::vector) into a binary format and then correctly deserialise it and be able to read the data. This is my first time using a binary format (I was using ASCII but that has become too hard to use now) so I am starting simple with just a vector of ints.

Whenever I read the data back the vector always has the right length but the data is either 0, undefined or random.

class Example  
{  
public:  
    std::vector<int> val;  
};

WRITE:

Example example = Example();  
example.val.push_back(10);  
size_t size = sizeof BinaryExample + (sizeof(int) * example.val.size()); 

std::fstream file ("Levels/example.sld", std::ios::out | std::ios::binary);

if (file.is_open())  
{  
    file.seekg(0);  
    file.write((char*)&example, size);  
    file.close();  
}

READ:

BinaryExample example = BinaryExample();

std::ifstream::pos_type size;  
std::ifstream file ("Levels/example.sld", std::ios::in | std::ios::binary | std::ios::ate);

if (file.is_open())  
{   
    size = file.tellg();

    file.seekg(0, std::ios::beg);
    file.read((char*)&example, size);
    file.close();
}

Does anyone know what I am doing wrong or what to do or be able to point me in the direction that I need to do?

Welcome to SO. To format a code block, indent the whole code by 4 spaces. — kennytm, Aug 09 '10 at 07:47
Thanks for that, the formatting was driving me nuts. I have been talking to someone else about my question and it seems that I need to make multiple write calls for each element in the vector as well as the size of the vector. — Damon L, Aug 09 '10 at 07:50
You may want to spend some time reading http://stackoverflow.com/editing-help. — kennytm, Aug 09 '10 at 08:10

score 5 · Accepted Answer · edited May 08 '17 at 18:53

5

You can't unserialise a non-POD class by overwriting an existing instance as you seem to be trying to do - you need to give the class a constructor that reads the data from the stream and constructs a new instance of the class with it.

In outline, given something like this:

class A {
    A();   
    A( istream & is );    
    void serialise( ostream & os );
    vector <int> v;
};

then serialise() would write the length of the vector followed by the vector contents. The constructor would read the vector length, resize the vector using the length, then read the vector contents:

void A :: serialise( ostream & os ) {
    size_t vsize = v.size();    
    os.write((char*)&vsize, sizeof(vsize));
    os.write((char*)&v[0], vsize * sizeof(int) );
}

A :: A( istream & is ) {
    size_t vsize;
    is.read((char*)&vsize, sizeof(vsize));
    v.resize( vsize );
    is.read((char*)&v[0], vsize * sizeof(int));
}

edited May 08 '17 at 18:53

ngoldbaum

5,430
3
28
35

answered Aug 09 '10 at 07:50

@Neil: oops, I added part on ASCII format on the wrong entry. Hope you won't mind. I will remove my own answer. – kriss Aug 09 '10 at 08:00
@Neil: to be more precise, you can serialize (or memcopy) directly only trivial classes. – kriss Aug 09 '10 at 08:04
@kriss added note about non-POD to answer. – Aug 09 '10 at 09:05
@Neil: I would even word it 'non-POD and trivial', because you can't even manage all PODs structure that way (no dynamic structures or pointer allowed). – kriss Aug 09 '10 at 14:57
This is pretty much the way I went about it. Save the length then save the contents. Read the length then read the contents. – Damon L Aug 10 '10 at 03:17

score 2 · Answer 2 · answered Aug 09 '10 at 07:53

You're using the address of the vector. What you need/want is the address of the data being held by the vector. Writing, for example, would be something like:

size = example.size();
file.write((char *)&size, sizeof(size));
file.write((char *)&example[0], sizeof(example[0] * size));

score 2 · Answer 3 · answered Aug 09 '10 at 08:52

I would write in network byte order to ensure file can be written&read on any platform. So:

#include <fstream>
#include <iostream>
#include <iomanip>
#include <vector>

#include <arpa/inet.h>

int main(void) {

  std::vector<int32_t> v = std::vector<int32_t>();
  v.push_back(111);
  v.push_back(222);
  v.push_back(333);


  {
    std::ofstream ofs;
    ofs.open("vecdmp.bin", std::ios::out | std::ios::binary);

    uint32_t sz = htonl(v.size());
    ofs.write((const char*)&sz, sizeof(uint32_t));
    for (uint32_t i = 0, end_i = v.size(); i < end_i; ++i) {
      int32_t val = htonl(v[i]);
      ofs.write((const char*)&val, sizeof(int32_t));
    }

    ofs.close();
  }

  {
    std::ifstream ifs;
    ifs.open("vecdmp.bin", std::ios::in | std::ios::binary);

    uint32_t sz = 0;
    ifs.read((char*)&sz, sizeof(uint32_t));
    sz = ntohl(sz);

    for (uint32_t i = 0; i < sz; ++i) {
      int32_t val = 0;
      ifs.read((char*)&val, sizeof(int32_t));
      val = ntohl(val);
      std::cout << i << '=' << val << '\n';
    }
  }

  return 0;
}

can you please help me here https://stackoverflow.com/q/73953638/18230141 ? — James_sheford, Oct 05 '22 at 08:19

score 1 · Answer 4 · answered Aug 09 '10 at 11:20

1

Read the other's answer to see how you should read/write a binary structure.

I add this one because I believe your motivations for using a binary format are mistaken. A binary format won't be easier that an ASCII one, usually it's the other way around.

You have many options to save/read data for long term use (ORM, databases, structured formats, configuration files, etc). The flat binary file is usually the worst and the harder to maintain except for very simple structures.

answered Aug 09 '10 at 11:20

kriss

23,497
17
97
116

The way that I was using ASCII was easy, but not very modular or extendible. I haven't done binary files before so I thought I would give that a go, thanks for the comment. – Damon L Aug 10 '10 at 03:15

Serialize and deserialize vector in binary

4 Answers4

Linked

Related