C++ struct serialization

Question

I'm implementing a data buffer which receives audio data packages with procedure call (no network protocols just two applications running on same machine) from one application and puts it in a Struct and writes to a mapped file.

So the writer application may call my app's procedure, which would be smth like void writeData (DataItem data, Timestamp ts) for about 15 times a second with each data item size 2MB.

My app shall store the data into a struct like

Struct DataItem 
{
long id;
...  Data; 
Time  insertTime;
}

and write it to a file for future reading purposes.

So since its hard to save the struct to the file as it is, I think(?) I need to write it as binary. So I'm not sure that I need to use any kind of serialization like boost serialization or not?

And I don't know how to align this data for memory map files, and how to re-construct the data for reading purpose from the file as well.

I search internet but I couldn't find much code example. And sample code would be higly appriciated.

By the way I'm using Windows 7 x64 embedded and Visual Studio 2008.

Thanks...

however you align it you just need to read it correctly in the other app: This post might help (http://stackoverflow.com/questions/8329767/writing-into-binary-files) — Gasim, Jan 27 '14 at 08:42
In C++ all keywords are written lower case. Its "struct" not "Struct" and a semi colon after the "}" is required. — RED SOFT ADAIR, Jan 27 '14 at 11:13
Yeah I wrote it below; it was just a pseudo, to give the idea what i am thinking, so don't mind about the syntax for now. — Vecihi, Jan 27 '14 at 11:30
If you title and tag your Question "C++" it might be helpful to adhere to correct syntax. It will give a better impression, less confusion and raises the probability of somebody answering your question. — RED SOFT ADAIR, Jan 27 '14 at 13:05

RED SOFT ADAIR · Answer 1 · 2014-01-27T14:29:34.853

A common C++ way to serialize would be:

struct myStruct
{
    int         IntData;
    float       FloatData;
    std::string StringData;
};

std::ostream& operator<<(std::ostream &os, const myStruct &myThing)

{
    os
    << myThing.IntData      << " "
    << myThing.FloatData    << " "
    << myThing.StringData   << " "
    ;
    return os;
}

std::istream& operator>>(std::istream &is, myStruct &myThing)
{
    is 
    >> myThing.IntData
    >> myThing.FloatData
    >> myThing.StringData;

    return is;
}


void WriteThing()
{
    myStruct myThing;

    myThing.IntData = 42;
    myThing.FloatData = 0.123;
    myThing.StringData = "My_String_Test";

    std::ofstream outFile;
    outFile.open("myFile.txt");

    outFile << myThing;
}

void ReadThing()
{
    myStruct myThing;

    std::ifstream inFile;
    inFile.open("myFile.txt");    
    inFile >> myThing;
}

Please Note:

std::string defines operators << and >>. Those will be called in the code above.
streams will treat white space characters as delimiters. Storing Strings with blanks would require additional handling
If you plan to keep your data through updates of your software, you must implement some sort of file versioning
refer to the docs of fstream to find out how to move the file pointer using seek etc. on a single large file.

score 0 · Answer 2 · answered Jan 27 '14 at 15:40

0

Use boost::serialization with text archive.
Is the most "standard" way of solving platform independence.
Optional, you can set a gzip compression on top of it.

answered Jan 27 '14 at 15:40

vlg789

751
6
20

GarMan · Answer 3 · 2014-01-27T08:42:36.453

-1

Are you sure you are asking about C++ and not C#? Your code example looks like C#

In C++ If your struct format is not going to change, then you can just write the array out to disk.

here is an example as you requested, but this is really C 101 stuff

FILE* output=fopen ("myfile", "wb");
fwrite (array, sizeof (mystruct), number_of_elements_in_array, output);
fclose (output);

edited Jan 27 '14 at 08:42

answered Jan 27 '14 at 08:33

GarMan

1,034
7
10

it is C++, I just wrote pseudo, didn't think about the syntax. How to write and read it directly to disk? Any example? – Vecihi Jan 27 '14 at 08:36
The performance is so important in my case. So opening and closing the file everytime slows down the app too much. Because of that I'm trying to implement memory mapped files. Create a 100GB file on disk map some portion to memory, write the data there and change the portion. So do you think that I can directly write and read the structs to memory (and then to the file) without any need of serialization? And also is it possible to search the data according to id attribute in the file? – Vecihi Jan 27 '14 at 08:53
"you can just write the array out to disk" - yes. But your example assumes that `Time` is POD, as well as all other types. Without the warning, this is very dangerous advice. Don't conflate C and C++ (as C has only POD structs) – sehe Jan 27 '14 at 11:23
@Vecihi for that, look at Boost Interprocess – sehe Jan 27 '14 at 13:11

C++ struct serialization

3 Answers3