Using packed bitfields for non-power-of-two byte integral members of a structure

Question

I have some binary data that contains structures that are defined as follows:

s1:
a - 1B
b - 4B
c - 2B
d - 8B

s2:
a - 1B
b - 3B
c - 2B
d - 6B

It uses Big Endian byte order.

I parse s1 with the following code:

#include <endian.h>
#include <stdint.h>
#include <string.h>

struct [[gnu::packed]] s1 {
    uint8_t  a;
    uint32_t b;
    uint16_t c;
    uint64_t d;
};

void parse_s1(struct s1 *parsed, const unsigned char buf[static restrict sizeof(*parsed)])
{
    memcpy(parsed, buf, sizeof(*parsed));
    parsed->b = be32toh(parsed->b);
    parsed->c = be16toh(parsed->c);
    parsed->d = be64toh(parsed->d);
}

Which AFAIK, is defined behavior.

And for the second structure, I'm considering the following code, but am not sure if I may be hitting Undefined Behavior at some point (I was lucky that the fields are byte-aligned (i.e., no 3-bit field) and that the non-power-of-two byte fields are not consecutive, but a complete answer might want to consider what would happen if two non-power-of-two byte integers are consecutive (i.e., if d went before c)):

#include <endian.h>
#include <stdint.h>
#include <string.h>

struct [[gnu::packed]] s2 {
    uint8_t   a;
    uintmax_t b : 24;
    uint16_t  c;
    uintmax_t d : 48;
};

void parse_s2(struct s2 *parsed, const unsigned char buf[static restrict sizeof(*parsed)])
{
    memcpy(parsed, buf, sizeof(*parsed));
    parsed->b = be32toh(parsed->b) >> 8;
    parsed->c = be16toh(parsed->c);
    parsed->d = be64toh(parsed->d) >> 16;
}

I also assume this code is C++-compatible (or more specifically gnu++-compatible) (except for the actual prototype of the function, which in C++ would use __restrict__ and a pointer instead of a VLA).

Is the code above correct in both C and C++ (GNU dialects)? Or does it rely on Undefined Behavior?

---

EDIT:

The following is an answer to @Anaconda's comment, as this doesn't format well in a comment:

//bitfields.c

#include <stdint.h>
#include <stdio.h>

struct [[gnu::packed]] s2 {
    uint8_t   a;
    uintmax_t b : 24;
    uint16_t  c;
    uintmax_t d : 48;
};

int main(void)
{
    printf("%zu\n", sizeof(struct s2));
    return 0;
}

$ cc -Wall -Wextra -pedantic -std=c2x bitfields.c

$ ./a.out
12

It seems (experimentally) that GCC understands [[gnu::packed]] so that it compresses bitfields as much as it can (at least byte-wise; I don't deal with integer widths that aren't multiple of 8, so I'm fine with this).

I'm not sure if there's any UB here, but I guess I'd have to be really unlucky, and GCC really evil, to break the code above.

Answer 1

Is the code above correct in both C and C++ (GNU dialects)? Or does it rely on Undefined Behavior?

Well, it's UB according to the standard.

According to GNU, all these details are "Determined by ABI".

According to Agner Fog's summary of x86 calling conventions [pdf],

Objects of structures and classes are stored by placing the data members consecutively in memory. Unused bytes may be inserted between elements and after the last element, if needed, for the sake of alignment. The requirements for alignment are ...

There's no language permitting reordering, nor special language about bitfields. So, that should broadly cover GNU on x86. If you want GNU on a different platform, either say what it is or look up its ABI doc.

---

NB. other language features - specifically pointers and C++ references - may work badly or not at all with misaligned and oddly-sized data members.

It's sometimes more efficient to deserialize into a normally-aligned structure the slow-looking way, operate on correctly-aligned normally-sized types, and re-serialize the result if necessary.

Answer 2

An experimental answer on x86_64 (just compile the same code on other architectures to test them):

// bitfields.c:

#define _DEFAULT_SOURCE
#include <endian.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>

struct [[gnu::packed]] s2 {
    uint8_t   a;
    uintmax_t b : 24;
    uint16_t  c;
    uintmax_t d : 48;
};

uint32_t be24toh(uint32_t be)
{
#if BYTE_ORDER == BIG_ENDIAN
    return be;
#elif BYTE_ORDER == LITTLE_ENDIAN
    return be32toh(be) >> 8;
#else
    #error "wtf"
#endif
}

uint64_t be48toh(uint64_t be)
{
#if BYTE_ORDER == BIG_ENDIAN
    return be;
#elif BYTE_ORDER == LITTLE_ENDIAN
    return be64toh(be) >> 16;
#else
    #error "wtf"
#endif
}

void parse_s2(struct s2 *s, const unsigned char *raw)
{
    memcpy(s, raw, sizeof(*s));
    s->b = be24toh(s->b);
    s->c = be16toh(s->c);
    s->d = be48toh(s->d);
}

int main(void)
{
    struct s2 s;
    unsigned char raw[sizeof(s)] = {1,2,3,4,5,6,7,8,9,0xA,0xB,0xC};

    parse_s2(&s, raw);

    puts("sizeof:");
    printf("s:   %zu\n", sizeof(s));
    printf("s.a: %zu\n", sizeof(s.a));
    printf("s.c: %zu\n", sizeof(s.c));

    puts("offsetof:");
    printf("s.a: %zu\n", offsetof(struct s2, a));
    printf("s.c: %zu\n", offsetof(struct s2, c));

    puts("contents:");
    printf("s.a: %#.2jx\n", (uintmax_t) s.a);
    printf("s.b: %#.6jx\n", (uintmax_t) s.b);
    printf("s.c: %#.4jx\n", (uintmax_t) s.c);
    printf("s.d: %#.12jx\n", (uintmax_t) s.d);
    return 0;
}

Results:

$ cc -Wall -Wextra -pedantic -std=c2x bitfields.c 
$ ./a.out 
sizeof:
s:   12
s.a: 1
s.c: 2
offsetof:
s.a: 0
s.c: 4
contents:
s.a: 0x01
s.b: 0x020304
s.c: 0x0506
s.d: 0x0708090a0b0c

$ c++ -Wall -Wextra -pedantic -std=c++20 bitfields.c 
$ ./a.out 
sizeof:
s:   12
s.a: 1
s.c: 2
offsetof:
s.a: 0
s.c: 4
contents:
s.a: 0x01
s.b: 0x020304
s.c: 0x0506
s.d: 0x0708090a0b0c

It works as expected on GCC 11 (x86_64) and doesn't trigger any warnings.