1

I have read about problems with loading structs from file. There are problems with endianness and different variable sizes. But let us say that there is a structure like this one:

struct Structure
{
    uint8_t value1;
    uint16_t value2;
    uint32_t value3;
    uint64_t value;4
};

Let us say that the file is always written in little-endian format, so application reads it in strict way. In such case endianness should not cause any problems. (Let us assume that there is some kind of convertEndinness() function which is clever enough to omit byte order issue). The second thing which I know is neccesery to consider is variable size variety. There is my question. Do fixed size types manage to handle this problem and what else should I consider in order to create multiplatform binary file?

Akasata Akasata
  • 333
  • 4
  • 10
  • What does "endianness doesn't matter because of specific file format." mean? – divinas Jul 03 '18 at 11:32
  • I'm sorry for not being exact enough. Let's say that file is always written in little endian format. For instance if platform is big endian compatible, application converts data to little endian during save operation. – Akasata Akasata Jul 03 '18 at 11:35
  • @AkasataAkasata I have reworded the question to use your last comment. The question is now much clearer (and less likely to be closed for "unclear what you are asking"), but it is now *so* obvious that I am not sure it is actually what you meant. If not, could you revert my change, and try to reword the question to ask what you *do* mean. – Martin Bonner supports Monica Jul 03 '18 at 11:41
  • It's *safe* to load, in the sense of being well-defined, but you need to adjust the endianness afterwards if necessary. – molbdnilo Jul 03 '18 at 11:46
  • 1
    You should also consider struct packing if you think about a multiplatform binary. If a file is transferred between architectures, you will face cases when different compilers produce different structure sizes. – Sianur Jul 03 '18 at 12:02
  • Sure. But what if I had binary file consisted of fixed size variables, without any structures and classes, would it be cross-platform file? – Akasata Akasata Jul 03 '18 at 12:06
  • If this question is related to something you actually need to do, rather than just out of curiosity, then please just use a proper serialisation method to store things. – Sean Burton Jul 03 '18 at 14:10

1 Answers1

2

Do fixed size types manage to handle this problem

Not quite.

The fixed-size types have fixed sizes, but their alignment requirements (and therefore padding) may vary between platforms and/or ABI flavours. So, your struct could still have different layout on different platforms even with the same endianness.

You can insist that there should be no padding, and use some compiler-specific and non-standard way to specify this (like #pragma pack or __attribute__((packed))). This can produce worse code for accessing misaligned members directly, though.

what else should I consider in order to create multiplatform binary file?

If you choose an endianness, use fixed-size types and specify the alignment correctly, you're probably fine.

I'd strongly suggested adding a header and/or some framing information, with a version and possibly some metadata about the sizes and alignments you chose. Otherwise you can never change this file format in the future without things breaking in unpleasant ways.

Useless
  • 64,155
  • 6
  • 88
  • 132
  • Thank you for your response, but what if I had binary file consisted of fixed size variables, without any structures and with strict endianness, would it be cross-platform file? – Akasata Akasata Jul 03 '18 at 12:16
  • Not necessarily, but it will _probably_ be fine. In reality if you're persisting a plain array of `uint32_t` or `uint64_t` or something, you're unlikely to get padding - although you should use `static_assert` to verify this assumption if you're going to depend on it. – Useless Jul 03 '18 at 13:07
  • So the only way is probably to load data byte by byte, not whole arrays? – Akasata Akasata Jul 03 '18 at 13:10
  • As I say, it's _probably_ fine, but not guaranteed to be portable. So, _how_ portable does it really need to be? Something that works for all x86 platforms may not also work for other architectures. Does this matter to you? – Useless Jul 03 '18 at 13:14
  • To be honest, I want to make binary file that works on windows, linux, mac, both x86 and x64. I come up with idea that I can check if file is written in same endianness at the very beggining of application and then if it's not i would be converted to proper one. Later application would use file in its own endianness. – Akasata Akasata Jul 03 '18 at 13:22
  • If you're only using x86(_64), you only have one endianness, and don't need to worry about this at all. – Useless Jul 03 '18 at 13:59
  • So conversion method should work properly in this case? – Akasata Akasata Jul 03 '18 at 14:17
  • Yep. Be aware that it's neither standard nor entirely portable - it's just portable enough for your use case, in practice. BTW I'm being prompted to move this discussion to chat, which is usually a sign that the question was under-specified in the first place. If you want to edit your question to include all the details added in comments, it'll be easier for others to read. – Useless Jul 03 '18 at 14:53
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/174271/discussion-between-akasata-akasata-and-useless). – Akasata Akasata Jul 03 '18 at 15:00