Size mismatch between Python and C struct with default struct alignment/padding

Question

I have a struct that I'm sending from a C server to a Python client. The C struct is the following one:

// Data structure to be sent to python interface
typedef struct {                                        //
    uint32_t componentType;                             // 4
    bool componentId_present;                           // 1 + 3 padding = 4
    uint32_t componentIdType;                           // 4 + 4 padding = 8
    long componentId;                                   // 8
    uint32_t componentConfigUpdate_type;                // 4 + 4 padding = 8
    bool componentConfigUpdate_s1AP_present;            // 1 + 3 padding = 4
    uint32_t componentConfigUpdate_s1AP_size;           // 4
    byte componentConfigUpdate_s1AP[128];               // 128
    bool componentConfigUpdate_x2AP_present;            // 1 + 3 padding = 4
    uint32_t componentConfigUpdate_x2AP_size;           // 4
    byte componentConfigUpdate_x2AP[128];               // 128
} data_E2setupRequest_NodeComponentConfigUpdate_t;      // 256 + 3*8 + 6*4 = 256 + 24 + 24 = 304

In Python I compute the size to be received with the following code:

import struct
size = struct.calcsize("i?ili?i128s?i128s")             # returns 300

As you can see sizes are different: 304 bytes vs 300 bytes. I have read this on stackoverflow and The Lost Ark of Structure Packing, but I cannot explain why there is such difference with default padding/packing rules.

Anyway I solved by setting the struct in this way (long var one place before):

typedef struct {
    uint32_t componentType;                             // 4
    bool componentId_present;                           // 1 + 3 padding = 4
    long componentId;                                   // 8
    uint32_t componentIdType;                           // 4 + 0 padding = 4
    uint32_t componentConfigUpdate_type;                // 4 + 0 padding = 4
    bool componentConfigUpdate_s1AP_present;            // 1 + 3 padding = 4
    ....
} data_E2setupRequest_NodeComponentConfigUpdate_t;      // 256 + 8 + 8*4 = 256 + 8 + 32 = 296

and

import struct
size = struct.calcsize("i?lii?i128s?i128s")             # returns 296

On which architecture(s)? (32/64bit, endianness...) – Mathieu Feb 26 '21 at 09:16 — Mathieu, Feb 26 '21 at 09:16
I'm compiling from a little endian x86_64 GNU/Linux – Aeglas Feb 26 '21 at 11:25 — Aeglas, Feb 26 '21 at 11:25

user2357112 · Accepted Answer · 2021-02-26T10:02:27.263

From the struct module docs:

Padding is only automatically added between successive structure members. No padding is added at the beginning or the end of the encoded struct.

You've miscalculated the padding for your C structure - assuming typical structure layout and 8-byte longs, sizeof(data_E2setupRequest_NodeComponentConfigUpdate_t) will be 304 rather than 300, but componentConfigUpdate_s1AP_present actually goes in the space you thought was padding after componentConfigUpdate_type. The extra 4 bytes of padding are actually at the end of the structure, and struct doesn't add that padding.

Also from the struct module docs:

To align the end of a structure to the alignment requirement of a particular type, end the format with the code for that type with a repeat count of zero.

So if you want struct to pad the end of the structure to long alignment, you can add 0l to the end of the format string.

(Also, bool is 1 byte, not 4 - there are 3 bytes of padding after all the bools in your struct. You could get rid of some of that padding by putting all the bools next to each other.)

Size mismatch between Python and C struct with default struct alignment/padding

1 Answers1