Reading and writing structs to binary file if struct has changed

Question

I have the following struct

struct MyStruct
{
    int     param1;
    float   param2;
    double  param3;
}

which I can write to a binary file using

fstream binary_file(file, ios::out|ios::binary); 
binary_file.seekg(0, ios::beg);
binary_file.write((char *)aStruct,sizeof(MyStruct));
binary_file.close();

and I can recover it using

ifstream binary_file;
binary_file.open(file, ios::binary);
binary_file.seekg(0, ios::beg);
binary_file.read((char *)aStruct, sizeof(MyStruct));
binary_file.seekg (0, ios::end);
binary_file.close();

This all works fine. Now change the defintion of the struct to

struct MyStruct
{
    int     param1;
    float   param2;
    double  param3;
    int     paramA;
    float   paramB;
    double  paramC
}

The question is, if I read a file which was written before the definition change, will param1, param2 and param3 always be correctly set and can I be sure that paramA, paramB and paramC will not be assigned any junk? Parameters will only be added to the end of the struct.

According to the reference the ifstream read function should stop if eof is encountered before having read the specified number of bits, so hopefully this is as easy as it sounds. My tests also indicate that the answer to the question would be yes, however I want to make sure with you guys as I have been reading about for example padding in binary files and don't completely understand how that works.

The results of my tests were that param1, param2 and param3 were set correctly and the values of paramA, paramB and paramC were 0, 0.0f and 0.0 repsectively. However, I just wanted to make sure that this was not just a coincidence. — joaerl, Jun 05 '13 at 16:00
Look at Google's [Protocol Buffers](http://code.google.com/p/protobuf/) for code which can handle versioning of structures in contexts such as this. — Jonathan Leffler, Jun 05 '13 at 16:15

Sergey Kalinichenko · Accepted Answer · 2013-06-05T16:17:14.570

4

The question is, if I read a file which was written before the definition change, will param1, param2 and param3 always be correctly set

Yes, you can be sure of that.

can I be sure that paramA, paramB and paramC will not be assigned any junk?

That is also correct, with one caveat: unless these fields are initialized in the constructor, they would remain uninitialized after the read (i.e. contain "junk").

Finally, note that this trick would work only for reading a single struct from the binary file. If you need to save an array of such structs, you would have to store the sizeof that was in effect at the time of writing out the data, otherwise you would not be able to partition the array correctly.

Obviously, changing to a different compiler or to a different version of the same compiler could also this scheme even without adding new fields to the end of the struct.

edited Jun 05 '13 at 16:17

answered Jun 05 '13 at 15:45

Sergey Kalinichenko

714,442
84
1,110
1,523

Thank you for your fast response. If I do not specify a constructor for my struct, is there a "default" constructor that will initialize the members? The following post seem to indicate that, http://stackoverflow.com/questions/8280023/does-a-c-struct-have-a-default-constructor – joaerl Jun 05 '13 at 15:58
I don't think this is safe. If the new attributes caused the padding of the structure to change the first attributes could wind up with garbage. – Mark B Jun 05 '13 at 16:01
1

@joaerl The answer at the link says that you'd get a default constructor, but it also says that the constructed fields may remain uninitialized, depending on their type. In your case, the fields will definitely remain uninitialized, because they are of primitive C++ types. – Sergey Kalinichenko Jun 05 '13 at 16:10
@MarkB I don't think that the standard lets compilers change padding between the fields ahead of the newly added ones; only the padding after the last of the old fields and the first of the newly added fields can change. This additional padding will have no effect, because there will be no data in the file that would go into that new "gap". – Sergey Kalinichenko Jun 05 '13 at 16:15
@dasblinkenlight consider a slightly different struct: `struct test { int a; int b; int c; };` changed to `struct test2 { int a; int b; int c; double d; };`. Are you suggesting that the compiler will leave `d` mis-aligned in `test2`? – Mark B Jun 05 '13 at 16:43
1

@MarkB Absolutely not! I'm suggesting that the gaps, if any, between the original members `a`, `b`, and `c` would remain unchanged. The gap between them and `d` has no effect on the code reading the data, because there wasn't enough data in the original `struct` to put stuff into this gap. – Sergey Kalinichenko Jun 05 '13 at 16:45

score 0 · Answer 2 · answered Jun 05 '13 at 16:09

This might work for you or it might not.

If the compiler changes the padding of the original attributes when you add the new ones, it won't be able to read the original values.

If the padding (and endian-ness) doesn't change then you'll most likely be able to successfully recover a single receord from the file with simply uninitialized data in the new attributes.

Reading and writing structs to binary file if struct has changed

2 Answers2