Byte vs bit order in endianess

Question

I know that the byte order things are relevant when we talk about network data transmission, i.e. network byte order (big-endian) is not the native thing for our operation systems, so bytes should be converted (swapped). E.g. if we have uint32_t with bytes 0x41 0x42 0x43 0x44 in memory in little-endian (OS-native) representation, for sending it through the network we must swap it to 0x44 0x43 0x42 0x41.

But what is the problem with bits and bitfields?? Why bits inside one byte (uint8_t) can / need to be swapped while network communication?

Example from DPDK:

__extension__                                                                                                                    
struct rte_gtp_hdr {                                                                                                 
        union {                                                                                   
                uint8_t gtp_hdr_info; /**< GTP header info */                                                  
                struct {                                                                           
#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN                                                                                                      
                        uint8_t pn:1;   /**< N-PDU Number present bit */                                                                     
                        uint8_t s:1;    /**< Sequence Number Present bit */                                                                  
                        uint8_t e:1;    /**< Extension Present bit */                                                                        
                        uint8_t res1:1; /**< Reserved */                                                              
                        uint8_t pt:1;   /**< Protocol Type bit */                                 
                        uint8_t ver:3;  /**< Version Number */                                                                               
#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN                                                                       
                        uint8_t ver:3;  /**< Version Number */                                    
                        uint8_t pt:1;   /**< Protocol Type bit */                                                                            
                        uint8_t res1:1; /**< Reserved */                                          
                        uint8_t e:1;    /**< Extension Present bit */                                          
                        uint8_t s:1;    /**< Sequence Number Present bit */                        
                        uint8_t pn:1;   /**< N-PDU Number present bit */                                                                     
#endif                                                                                                                                       
                };                                                                                                                           
        };                                                                                                                                   
        uint8_t msg_type;     /**< GTP message type */                                                                                       
        rte_be16_t plen;      /**< Total payload length */                                                                                   
        rte_be32_t teid;      /**< Tunnel endpoint ID */                                                                                     
} __rte_packed;

Why in this specific example bits order in bitfield depends on RTE_BYTE_ORDER (keyword byte)? Bit order depends on byte order? O_o

One more example in Linux kernel.

Sorry if obvious..

Read this: [**Order of fields when using a bit field in C**](https://stackoverflow.com/questions/19376426/order-of-fields-when-using-a-bit-field-in-c): "C standard allows compiler to put bit-fields in any order. ..." I'll let others decide if this question is a duplicate of that one.... — Andrew Henle, Aug 17 '23 at 18:20
See [this](https://stackoverflow.com/questions/18070977/little-endian-bitfield-and-big-endian-bitfield) and [this](https://stackoverflow.com/questions/6043483/why-bit-endianness-is-an-issue-in-bitfields) question, it seems to be already answered. Also here is an explanation with pictures: [link](http://mjfrazer.org/mjfrazer/bitfields/). — Sam Protsenko, Aug 17 '23 at 18:55
Does this answer your question? [Order of fields when using a bit field in C](https://stackoverflow.com/questions/19376426/order-of-fields-when-using-a-bit-field-in-c) - The linked answers together with the external article @SamProtsenko shared should be good enough. Removed my answer ... — Ted Lyngmo, Aug 17 '23 at 19:05
This configuration is for a specific compiler. If you use a different compiler, like on Windows, you can (and will) get a different order for the bit fields. But you just don't compile the Linux kernel there. — BoP, Aug 17 '23 at 19:11

score 1 · Answer 1 · answered Aug 17 '23 at 20:25

1

But what is the problem with bits and bitfields?? Why bits inside one byte (uint8_t) can / need to be swapped while network communication?

Individual bits are not addressable in C. The bits within the same byte do not have any inherent relative order -- only whatever order we attribute to them externally according to their place values or some other criterion.

Individual bits do have a relative order in serial binary communications media. Protocols that operate over such media define the transformation between byte stream and bit stream, but the actual conversion is generally performed in hardware, not software.

However, I think you're looking at something different. It's unlikely to be about bit order so much as about bitfield mapping -- that is, how C implementations lay out storage for bitfield members within a struct. C leaves some significant details of that mapping at the discretion of implementations. It is not inherently tied to the architecture's native endianness.

Some network protocols (for example) use data structures that include fields narrower than one byte. If you want to map that to C bitfields then you need to account for multiple factors, including how your particular C implementation lays out bitfields within a struct. That's what I think you are seeing.

And note well that I see no reason to think that any swapping is going on in your example. It's just that the C structure definition needs to be different on some C implementations than on others to compensate for implementations' different choices, so that the underlying sequences of bytes of the structures' representations are correct for their purpose.

answered Aug 17 '23 at 20:25

John Bollinger

160,171
8
81
157

Thanks! The fist question is about *"It's just that the C structure definition needs to be different on some C implementations*" - both DPDK and Linux kernel compiles with specific compilers and even compiler versions. Why do they care about different implementations? – NK-cell Aug 18 '23 at 12:32
The second question (if you don't mind) - does pragma pack affect bitfields order within one byte? – NK-cell Aug 18 '23 at 12:35
@NK-cell, I can't speak to DPDK specifically, but certainly various versions of the Linux kernel have been targeted for building with numerous C implementations over the years, even if they have all been versions of GCC for various architectures. The particular part you are looking at may well be in response to an actual issue that was encountered at some point in the kernel's history. Otherwise, it may simply be forward looking -- possibly even to the point of overengineering. – John Bollinger Aug 18 '23 at 13:50
@NK-cell, one question per question, please. – John Bollinger Aug 18 '23 at 13:50
Looks strange that both DPDK and Linux kernel do overengineering in protocol headers. As well as they had some similar historical problems in their code, which caused this bitfield acrobatics. Don't you think so? – NK-cell Aug 18 '23 at 14:17
No, @NK-cell, I do not think it particularly strange that projects with a long history and wide distribution to diverse hardware might have run into and solved *numerous* issues revolving around platform-dependent behavior. The Linux kernel in particular has routine exposure to that sort of thing, but userspace software is by no means immune. And I don't think it particularly strange that a developer of software sensitive to such issues might address them preemptively. – John Bollinger Aug 18 '23 at 14:41

Byte vs bit order in endianess

1 Answers1