Endianess inside a byte

Question

Recently I'm tracking down a bug that appears when the two sides of the network communication have different endianness. One side has already sent a telegram marking lastSegment while the other side is still waiting for the last segment endlessly.

I read this code:

#ifndef kBigEndian
    struct tTelegram
    {
       u8 lastSegment : 1;
       u8 reserved: 7;
       u8 data[1];
    };
#else
    struct tTelegram
    {
       u8 reserved: 7;
       u8 lastSegment : 1;
       u8 data[1];
    };
#endif

I know endianness is concerned for multi-byte type, e.g., int, long, etc. But why it cares in the previous code? lastSegment and reserved are inside a single byte.

Is that a bug?

Bit-packing layout across compilers and architectures is not portable and is implementation-defined. You can't assume it will be done the same way. See also: http://stackoverflow.com/questions/1490092/c-c-force-bit-field-order-and-alignment — Joe, Dec 03 '12 at 04:13

score 2 · Answer 1 · answered Dec 03 '12 at 04:14

2

You have 16 bits in your struct. On a 32-bit or 64-bit architecture, depending on the endianess, data may come "before" reserved and lastSegment or it may come "after" when looking at the raw binary. IE If we consider 32 bits, your struct may be packed along 32-bit boundaries. It might look like this:

 padbyte1 padbyte2 data lastSegment+reserved

or it may look like this

 lastSegment+reserved data padbyte1 padbyte2

So when you put those 16 bits over the wire then reinterpret them on the other side, do you know if you're getting data or lastSegment?

Your problem isn't within the byte, its where data lies in relation to reserved and lastSegment.

answered Dec 03 '12 at 04:14

Doug T.

64,223
27
138
202

So I should change the position of endianness ifdef so that `lastSegment+reserved` comes before `data` in big endian and comes after `data` in little endian. Is that right? – Eric Z Dec 03 '12 at 04:36
2

@EricZ: No, you should avoid using bit-fields entirely if you care about having a well-defined data format. – jamesdlin Dec 03 '12 at 04:40

score 1 · Accepted Answer · answered Dec 03 '12 at 04:20

1

When it comes to bitfields, ordering is not guaranteed even between different compilers running on the same CPU. You could theoretically even get a change of order just by changing flags with the same compiler (though, in fairness, I have to add that I've never actually seen that happen).

answered Dec 03 '12 at 04:20

Jerry Coffin

476,176
80
629
1,111

So when passing a byte through network to the other side, the other side is not guaranteed to get the correct value? – Eric Z Dec 03 '12 at 04:33
@EricZ: That's right. Or more accurately, there really *is* no one "correct value" to receive. – Jerry Coffin Dec 03 '12 at 04:40
1

The other side will get the same byte that was sent... but the program receiving the byte may be compiled differently, so that it associates the struct's member variables with different bits than the program that sent the byte, and that would cause confusion/incompatibility even if the sender and the receiver use the same source code. If you avoid C's bit-field feature and instead use your own bit-packing logic (using bit-shifting and bitwise-and and bitwise-or) to pack/unpack bytes instead, you can avoid that particular gotcha. – Jeremy Friesner Dec 03 '12 at 04:41
But how does the general network protocol work? E.g., it usually only enforces the network byte order. But how does it guarantee that the other side will receive exactly the same data sent by the original side? You sent a message to your friend via MSN, e.g., it's likely that he'll received a different one? Confused. – Eric Z Dec 03 '12 at 04:44
@EricZ: At least if you send by TCP, you're pretty much guaranteed that what goes in one end of the wire will match what comes out the other end. The problem is entirely with how your CPU is programmed to interpret the bits. Jeremy is right: the primary way to avoid this is to manage the bit-packing and unpacking yourself. – Jerry Coffin Dec 03 '12 at 04:49
@Jeremy, thanks. If I only use one byte(no bit fields, just u8), and the sender sends `00000001`, will the other side always translate it back to `00000001`? Is there a chance that it's tranlated into `10000000`? – Eric Z Dec 03 '12 at 04:50
@Jerry, what if I'm using TCP/IP while the two sides interpret bits differently.e.g, due to different architecture, it's still possible that data could be interpreted wrong? – Eric Z Dec 03 '12 at 04:52
@EricZ: For a single byte, no, no change. They'll receive the same value you send. The network card takes care of that much. Endianess one comes into play when a value occupies more than one byte. – Jerry Coffin Dec 03 '12 at 04:55
@Jeffy, thanks. One last question: Generally, if I have one C/C++ structure containing several members(no bit fields, but u8, u16, etc. E.g., `struct SS {u8 m_a; u16 m_b};`), is the order of members(`m_a` comes before `m_b` or after) in structure interpreted the same on different architectures? – Eric Z Dec 03 '12 at 05:00
@EricZ: the order is always the same, but there may be differences in padding between them, so you generally need to control the padding explicitly for it to work. – Jerry Coffin Dec 03 '12 at 05:01

Endianess inside a byte

2 Answers2