How to do Endian conversion when dealing with bit fields

Question

I am dealing with a situation where I need to send/receive data via a TCP/IP socket between myself (client) and a server. The message structure is proprietary, but is basically arrays of uint32_t. I am tasked with handling the Endian conversion on my end. As the client, I am operating in Windows (little endian). The server is operating in VxWorks environment (big endian). Therefor, I need to convert data I send from little to big, and data I receive from big to little.

Now, I am aware that endianness refers to BYTE order within a word. So, I created a function that would do the byte swapping for each uint32_t word in a given array. See below.

void convertEndian(uint32_t inputData[], int size)
{
    uint32_t b1, b2, b3, b4;

    for (int i = 0; i < size; ++i)                                                               
    {
        b1 = (inputData[i] & 0xFF000000) >> 24;
        b2 = (inputData[i] & 0x00FF0000) >> 8;
        b3 = (inputData[i] & 0x0000FF00) << 8;
        b4 = (inputData[i] & 0x000000FF) << 24;
        
        inputData[i] = b1 | b2 | b3 | b4;
    }
}

This approach is fine for certain message types I'll be dealing with, where each word is defined by an entire uint32_t value. However, some messages have many words that have their own unique bit fields. Below is an example of one:

Struct test
{
                Unsigned int var1 : 16;
                Unsigned int var2 : 12;
                Unsigned int var3 : 1;
                Unsigned int var4 : 1;
                Unsigned int var5 : 1;
                Unsigned int var6 : 1;
}

How do I implement endian conversion for such cases? There is one message type for example, where I will be receiving an array of about 32 words of uint32_t and each of those words has its own set of bit fields representing various things.

I guess my only choice is to mask/shift for each word as needed. But then I will pretty much have to make 32 unique functions for each word. It seems very labor intensive.

Any suggestions would be appreciated.

Also see [this](https://stackoverflow.com/questions/19376426/order-of-fields-when-using-a-bit-field-in-c). — rustyx, May 26 '21 at 14:32
Could you confirm the layout of the bitfields in the message? There is a convention to lay out bitfields starting at the most-significant bit on Big Endian platforms, but it's not a given. — harold, May 26 '21 at 14:33
This is why people use network byte order, and why sending raw binary data over modern networks is far less common than it used to be. — Joseph Larson, May 26 '21 at 14:39
Related: [How to enforce the struct bit order with the gcc compiler](https://stackoverflow.com/questions/6728218/how-to-enforce-the-struct-bit-order-with-the-gcc-compiler) — gkhaos, May 26 '21 at 15:04
Can you add some data? Like 4 bytes you get vs 4 bytes you want? — gkhaos, May 26 '21 at 15:05
Which language, C or C++? They are distinct languages. For example, C++ has `std::bitset` and `std::vector`, the C language doesn't. — Thomas Matthews, May 26 '21 at 15:19
Not related to bit fields but... don't you have access to `ntoh` (and friends)? — Support Ukraine, May 26 '21 at 15:29
So the server part is also written by your company? which means you can agree on some format? — Surt, May 26 '21 at 15:40

score 0 · Answer 1 · answered May 26 '21 at 16:35

This is hopefully without too many assumptions.

Totally untested, humour not marked.

static_assert(sizeof(test) == sizeof (uint32_t), "Houston we have a problem");

template < typename NetOrHost>
void convertEndian(uint32_t inputData[], int size) {
  for (int i = 0; i < size; ++i)            
    inputData[i] = NetOrHost(inputData[i]);
}

// or simply use

std::for_each(inputData, inputData+size, htonl /* ntohl for the other way */); // this part needs a bit of testing ...

// Now encoding the test struct
// Using https://stackoverflow.com/a/20194422/4013258 BitCount
// or just use magic values, *hands OP a foot-gun*

// Once only
std::array<int, 6> bits; // todo make this constexpr
bits[0] = BitCount(test.var1);
bits[1] = BitCount(test.var2);
etc ...
static_assert(std::accumulate(bits.begin(), bits.end(), 0) == 32, "Preconditions violated");

// this goes well because we know that it fits in a 32-bit unsigned integer.
uint32_t encoded = 0;

encoded = test.var1;
encoded <<= bits[1];
encoded += test.var2;
encoded <<= bits[2];
encoded += test.var3;
etc.

// To decode
uint32_t decode = inputData[i];

test.var6 = decode & ((1 << bits[5])-1); // or make a mask array
decode >>= bits[5];
etc.

This would be a lot easier with reflection ...

I should note that this is for a C implementation, not C++ – TestEngineer11 Jun 02 '21 at 14:13 — TestEngineer11, Jun 02 '21 at 14:13

How to do Endian conversion when dealing with bit fields

1 Answers1