binary read/write arbitrary std::vector > in c++

Question

Hi, I would like to store binary a std::vector<std::vector<int> > object MATRIX in a file.

out.write((char*)&MATRIX, sizeof(MATRIX));

The problem is, that only the column dimension is fixed. The row dimension changes. If I read the object out of the binary file, it's not enough to know only the size, isn't it? So, initializing e.g. a second matrix with

std::vector<std::vector<int> > MATRIX2;
for ( int i=0;i<column_dim;i++ ) MATRIX2.push_back ( vector<int> ( 0 ) );
ifstream in(cstr, ios::in | ios::binary);

and reading the object data with

ifstream in(cstr, ios::in | ios::binary);    
in.read((char*)& MATRIX2, fSize);

makes no sense, because the compiler has no clue about the structure of the saved data. My question: Is there any better way of solving this problem than saving the matrix structure (all info about the row dimensions) in a second file, reading it and creating a MATRIX2 with the appropriate structure which is then filled by using

ifstream in(cstr, ios::in | ios::binary);    
in.read((char*)&nn_H_test, fSize);

?

Is the file format given? If not maybe you want to take a look at http://www.boost.org/doc/libs/1_46_1/libs/serialization/doc/index.html. This should make it quite easy for you to store and load a matrix of int. — mkaes, May 27 '11 at 12:44
Extend the protocol I suggested in http://stackoverflow.com/questions/3438132/serialise-and-deserialise-vector-in-binary/3438541#3438541, add one more size_t in front of each stored matrix to specify the first dimension. — bobah, May 27 '11 at 12:46
Ouch, are you trying to do a dump of a raw `std::vector` on file? That will *never* work, and reading it back you'll get an object in a completely messed up state (the only correct thing may be the vector size, but the internal pointer to the elements will be invalid). You need proper serialization. — Matteo Italia, May 27 '11 at 12:52
"That will never work" is not true. If I generate a MATRIX2 object with the same structure as MATRIX, everything is copied in the correct order. — pawel_winzig, May 27 '11 at 13:12
you are just being lucky, in general dumping a std::vector to file like that will result in dumping its private members, which usually are the vector size, the vector capacity and the *pointer* to the memory used by it. So, if you dump and reload it during the same execution it may work, but if you try to load it in a new process you will end with an invalid internal pointer. — Matteo Italia, May 27 '11 at 21:30

score 3 · Accepted Answer · answered May 27 '11 at 13:22

I will give you a short example on how to do this with boost.

#include <iostream>
#include <fstream>
#include <boost/serialization/serialization.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>

std::vector<std::vector<int > > g_matrix;
int main(int argc, char* argv[])
{
    // fill the vector
    std::ofstream ofs("c:\\dump");
    boost::archive::text_oarchive to(ofs);
    to << g_matrix;

    g_matrix.clear();
    std::ifstream ifs("c:\\dump");
    boost::archive::text_iarchive infs(ifs);
    infs >> g_matrix;
    // check your vector. It should be the same
}

If you need it to be more human readable you can also try the xml out from boost.
As always don't reinvent the wheel.

score 2 · Answer 2 · edited May 23 '17 at 12:13

2

ios::binary is almost certainly NOT going to have the result you think it will have - it only affects line ending translation.

Secondly, you are not going to escape having to store the individual lengths of the inner vectors if you want to restore the structure upon reading. Restoring a raw std::vector from a stream directly into a new std::vector like you are trying to now is not going to work, ever.

Structured restoring however need not be difficult, just store the number of values, and then all the values, for each inner vector. Your read routine can then safely read the first value, then read N next values, and assume the first value after that is the number of values of the next row.

edited May 23 '17 at 12:13

Community

1
1

answered May 27 '11 at 12:49

Joris Timmermans

10,814
2
49
75

Thanks for the comment. To your first remark: what would you suggest instead? – pawel_winzig May 27 '11 at 13:08
If you want to store the vector as true "binary" data (e.g. using 4 bytes on disk for a 4 byte integer) you should consider writing a custom binary file stream using stream read and write. – Joris Timmermans May 27 '11 at 13:35

score 0 · Answer 3 · answered May 27 '11 at 12:49

0

For something extremely simple why not just place a header value before your matrix output that specifies row column size? Trivial example:

4,2 //#rows, #columns
0 1
2 0
2 3
4 5

Now when you read in the matrix, read in the header information and then the matrix data.

If you want the matrix to be serializable, you should look at a serialization paradigm for your matrix structure. Read more about serialization in the C++ FAQ.

answered May 27 '11 at 12:49

nathan

5,513
4
35
47

Thanks, yes, this was also the first thing that came into my mind, but it has a drawback: Some of my algorithms use this matrix already I would have to rewrite them -- unfortunately this will cause more work that adding an additional file. – pawel_winzig May 27 '11 at 12:56
If that's a problem, then serialization is probably your solution. That way it's all self contained in the matrix. – nathan May 27 '11 at 13:16

binary read/write arbitrary std::vector > in c++

3 Answers3