Types bit length and architecture specific implementations

Question

I'm doing stuff in C++ but lately I've found that there are slight differences regarding how much data a type can accomodate and also the byte order is an issue.

Suppose I got a binary file, where I've encoded shorts that are 2 bytes in size. The file is in binary format like:

FA C8 - data segment 1

BA 32 - data segment 2

53 56 - data segment 3

Now all is well up to this point. Now I want to read this data. There are 2 problems:

1 what data type to choose to store this values? 
2 how to deal with endianness of the target architecture?

The first problem is actually related to the second because here I will have to do bit shifts in order to swap the order of bytes.

I know that I could read the file byte by byte and add every two bytes. But is there an approach that could ease that pain?

I'm sorry If I'm being ambiguous. The problem is hard to explain. Hope you get a glimpse of what I'm talking about. I just want to store this data internally.

So I would appreciate some advices or if you can share some of your experience in this topic.

score 2 · Answer 1 · answered May 26 '14 at 10:35

2

If you use big endian on the file that stores the data then you could just rely on htons(), htonl(), ntohs(), ntohl() to convert the integers to the right endianess before saving or after reading.

answered May 26 '14 at 10:35

Paolo Brandoli

4,681
26
38

I heard about this functions but can you tell where can I find them with GCC on MinGW? I had problem figuring out the correct headers, if there are any, at all. – luke1985 May 26 '14 at 10:39
On windows they are in Winsock2.h, on linux in #include . – Paolo Brandoli May 26 '14 at 10:41

score 1 · Answer 2 · edited May 23 '17 at 12:23

There is no easy way to do this.

Rather than doing that yourself, you might want to look into serialization libraries (for example Protobuf or boost serialization), they'll take care of a lot of that for you.

If you want to do it yourself, use fixed-width types (uint32_t and the like from <cstdint>), and endian conversion functions as appropriate. Either have a "prefix" in your file that determines what endianness it contains (a BOM/Byte Order Mark), or always store in either big or little endian, and systematically convert.

Be extra careful if you need to serialize strings, they have encoding problems of their own too.

Types bit length and architecture specific implementations

2 Answers2