Writing data structure with vector members to binary file

Question

Attempting to write a data struct containing vector members, I noticed that vector members were not accurate when the data was being read back from the buffer.

First I write out the structure manually with simple values: (Assume POD members are accurate in this case.)

void ObjFileImport::WriteGeometryFile(LPWSTR In_File){
BaseGeometry WriteTest;
WriteTest.VertexData = {
    XMFLOAT3(9.5, 6.7f, 9.3f),
    XMFLOAT3(3.5f, 6.7f, 9.3f),
    XMFLOAT3(3.5f, 6.7f, 9.3f) };
WriteTest.ColorData = { XMFLOAT4(3.5f, 6.7f, 9.3f, 5.6f) };
WriteTest.Update();

ofstream File;
File.open(In_File, ofstream::binary | ofstream::out);

if (File.is_open()){
    File.write((const char*)&WriteTest.Version, sizeof(float));

    File.write((const char*)&WriteTest.sz_Array[0], sizeof(size_t)*WriteTest.sz_BG_Mbr); 

    File.write((const char*)WriteTest.ColorData.data(),         sizeof(XMFLOAT4)*WriteTest.sz_ColorData);
    File.write((const char*)WriteTest.ImageData.data(),         sizeof(XMFLOAT4)*WriteTest.sz_ImageData);
    File.write((const char*)WriteTest.VertexData.data(),        sizeof(XMFLOAT3)*WriteTest.sz_VertexData);
    File.write((const char*)WriteTest.NormalData.data(),        sizeof(XMFLOAT3)*WriteTest.sz_NormalData);
    File.write((const char*)WriteTest.TextureCoordData.data(),  sizeof(XMFLOAT2)*WriteTest.sz_TextureCoordData);

    File.write((const char*)&WriteTest.IndexVertexData,         sizeof(UINT)*WriteTest.sz_IndexVertexData);
    File.write((const char*)&WriteTest.IndexTextureCoordData,   sizeof(UINT)*WriteTest.sz_IndexTextureCoordData);
    File.write((const char*)&WriteTest.IndexNormalData,         sizeof(UINT)*WriteTest.sz_IndexNormalData);
}
File.close();
}

This works fine... according to the regular types showing up on read, which looks like:

void ObjFileImport::ReadGeometryFile(LPWSTR In_File){
int Sz_Base = sizeof(BaseGeometry);
BaseGeometry Temp;
ifstream File;
File.open(In_File, ifstream::binary | ifstream::in | ifstream::ate);

if (File.is_open()){ 
    int FileSize = File.tellg();
    File.seekg(0, File.beg);

    //Read version number.
    File.read((char*)&Temp.Version, sizeof(float));

    File.read((char*)&Temp.sz_Array, sizeof(size_t)           *Temp.sz_BG_Mbr); //Test for heap tranfer

    File.read((char*)&Temp.ColorData, sizeof(XMFLOAT4)        *Temp.sz_ColorData);
    File.read((char*)&Temp.ImageData, sizeof(XMFLOAT4)        *Temp.sz_ImageData);
    File.read((char*)&Temp.VertexData, sizeof(XMFLOAT3)       *Temp.sz_VertexData);
    File.read((char*)&Temp.NormalData, sizeof(XMFLOAT3)       *Temp.sz_NormalData);
    File.read((char*)&Temp.TextureCoordData, sizeof(XMFLOAT2) *Temp.sz_TextureCoordData);

    File.read((char*)&Temp.IndexVertexData, sizeof(UINT)      *Temp.sz_IndexVertexData);
    File.read((char*)&Temp.IndexTextureCoordData, sizeof(UINT)*Temp.sz_IndexTextureCoordData);
    File.read((char*)&Temp.IndexNormalData, sizeof(UINT)      *Temp.sz_IndexNormalData);

    File.close();
}
}

Unfortunately, while reading from the data buffer, the vector values (and size) are not showing up.

I know that vector essentially points to data stored on the heap and tried to get the internal pointer for that data WriteTest.data() to read into the binary file. Upon reading out, the data doesn't seem to be structured intuitively.

Like you say, a vector contains a *pointer* to the heap-allocated data, and all heap-allocations (and, really, all pointers) are unique to the current process. If you save a pointer to allocated data, and then read it in another program (or even the same program but as a new process) then that pointer no longer points to allocated memory. You should search for and read about [*serialization*](https://en.wikipedia.org/wiki/Serialization). There are many good libraries to handle serialization for you, use them. — Some programmer dude, Mar 25 '16 at 02:05
If all of this stuff is a std::vector, why are there separate variables that indicate the size of each vector? Do you know about std::vecto::size()? And, when reading the data back, "File.read((char*)&Temp.ColorData," completely overwrites and scribbles over the `std::vector` itself, instead of its `data()`, and it is also your responsibility, upon reading, to size the std::vector so it contains sufficient elements, before reading the data into the std::vector's data(). — Sam Varshavchik, Mar 25 '16 at 02:10
Is there a way to reference the heap data directly - to copy to the file? — Rashid Ellis, Mar 25 '16 at 02:10
***Is there a way to reference the heap data directly - to copy to the file?*** No when it contains pointers you can not do that. Also the heap need not be contiguous and the objects you put on the heap need not be in any specific order. So you would get garbage back. — drescherjm, Mar 25 '16 at 02:13
You can access row data of vector by &v[0] http://stackoverflow.com/questions/6485496/how-to-get-stdvector-pointer-to-the-raw-data — VladimirS, Mar 25 '16 at 02:16
@SamVarshavchik The `BaseGeometry` struct is a -ahem- smart struct I created while trying to get the `vector` members to output. The `sz_...` members are `union`'ed to an array that act as a header, updated via `struct` function member, `update()`, giving the size of all arrays. This get passed in second successfully, and is used to structure the data to be read in... If things were easy. — Rashid Ellis, Mar 25 '16 at 02:24
@JoachimPileborg It keeps coming back to Serialization, I looked at http://www.boost.org/doc/libs/1_55_0/libs/serialization/doc/serialization.html before, and read up on it, but I was really opting for a less, seemingly invasive method. Maybe simpler. — Rashid Ellis, Mar 25 '16 at 02:35

score 1 · Accepted Answer · edited May 23 '17 at 12:15

1

C++ STL does not provide native interface to store STL object to disk.

You can check this thread for help.

edited May 23 '17 at 12:15

Community

1
1

answered Mar 25 '16 at 02:57

Wesley Ranger

770
1
7
26

score 0 · Answer 2 · answered Mar 25 '16 at 03:37

For this implementation, (As noted by Sam Varshavchik, comments) vector objects were being overwritten and [as well], the relevant data pointed to by the vector's internal pointer, when referenced, returned an assert error, due to not being sized correctly for the data being forced in.

void ObjFileImport::ReadGeometryFile(LPWSTR In_File){
int Sz_Base = sizeof(BaseGeometry);
BaseGeometry Temp;
ifstream File;
File.open(In_File, ifstream::binary | ifstream::in | ifstream::ate);

if (File.is_open()){ 
    int FileSize = File.tellg();
    File.seekg(0, File.beg);

    //Read version number.
    File.read((char*)&Temp.Version, sizeof(float));

    //Brace for horrible things later. Read size definition array.
    File.read((char*)&Temp.sz_Array, sizeof(size_t)*Temp.sz_BG_Mbr); 

    //If local pointer null, make not null, else, allocate space.
    if (Temp.sz_ColorData == 0)Temp.ColorData.resize(1);
    else Temp.ColorData.resize(Temp.sz_ColorData);

    if (Temp.sz_ImageData == 0)Temp.ImageData.resize(1);
    else Temp.ImageData.resize(Temp.sz_ImageData);

    if (Temp.sz_VertexData == 0)Temp.VertexData.resize(1);
    else Temp.VertexData.resize(Temp.sz_VertexData);

    if (Temp.sz_TextureCoordData == 0)Temp.TextureCoordData.resize(1);
    else Temp.TextureCoordData.resize(Temp.sz_TextureCoordData);

    if (Temp.sz_NormalData == 0)Temp.NormalData.resize(1);
    else Temp.NormalData.resize(Temp.sz_NormalData);

    if (Temp.sz_IndexVertexData == 0)Temp.IndexVertexData.resize(1);
    else Temp.IndexVertexData.resize(Temp.sz_IndexVertexData);

    if (Temp.sz_IndexTextureCoordData == 0)Temp.IndexTextureCoordData.resize(1);
    else Temp.IndexTextureCoordData.resize(Temp.sz_IndexTextureCoordData);

    if (Temp.sz_IndexNormalData == 0)Temp.IndexNormalData.resize(1);
    else Temp.IndexNormalData.resize(Temp.sz_IndexNormalData);

    //Read contents of buffer in.
    File.read((char*)Temp.ColorData.data(), sizeof(XMFLOAT4)          *Temp.sz_ColorData);       // *Temp.sz_ColorData);
    File.read((char*)Temp.ImageData.data(), sizeof(XMFLOAT4)          *Temp.sz_ImageData);
    File.read((char*)Temp.VertexData.data(), sizeof(XMFLOAT3)         *Temp.sz_VertexData);
    File.read((char*)Temp.NormalData.data(), sizeof(XMFLOAT3)         *Temp.sz_NormalData);
    File.read((char*)Temp.TextureCoordData.data(),   sizeof(XMFLOAT2) *Temp.sz_TextureCoordData);

    File.read((char*)Temp.IndexVertexData.data(),   sizeof(UINT)      *Temp.sz_IndexVertexData);
    File.read((char*)Temp.IndexTextureCoordData.data(),   sizeof(UINT)*Temp.sz_IndexTextureCoordData);
    File.read((char*)Temp.IndexNormalData.data(),    sizeof(UINT)     *Temp.sz_IndexNormalData);

    File.close();
}
}

The "Fixed" code, although whether the data "persists" across machine types may be up for question.

putting the data in a common endian will probably fix your persistence questions since this is a binary file. Although those functions are not std c++. — user2913685, Mar 25 '16 at 04:25

Writing data structure with vector members to binary file

2 Answers2