What's the correct (modern) way to pad a struct?

Question

edit: a better way of phrasing this: What's the correct [modern] way to ensure that a struct is a specific size in bytes?

just spending a relaxing saturday afternoon debugging a legacy codebase, and having a bit of trouble figuring this out. The compiler error I get is this:

INC/flx.h:33: error: dereferencing pointer to incomplete type

the code at line 33 looks like this

typedef struct flx_head {
FHEAD_COMMON;
LONG frames_in_table; /* size of index */
LONG index_oset;    /* offset to index */
LONG path_oset;     /* offset to flipath record chunk */

/* this will insure that a Flx_head is the same size as a fli_head but won't
 * work if there is < 2 bytes left (value <= 0) */

PADTO(sizeof(Fli_head),flx_head,flxpad);  /* line 33 is this one */
} Flx_head;

well okay so I can see that the struct is referring to itself to pad it out somehow. But I don't know an alternative way of doing what PADTO does without the self reference.

here's what PADTO is defined as

 #define MEMBER(struc,field) \
    ((struc*)NULL)->field

/* returns offset of field within a given struct name,
 * and field name ie: OFFSET(struct sname,fieldname) */

#define OFFSET(struc,field) \
    (USHORT)((ULONG)((PTR)&MEMBER(struc,field)-(PTR)NULL))

/* offset to first byte after a field */

#define POSTOSET(struc,field) \
    (OFFSET(struc,field)+sizeof(MEMBER(struc,field)))

/* macro for defining pad sizes in structures can not define a pad of 
 * less than two bytes  one may use pname for the offset to it but
 * sizeof(struc->pname) will not be valid 
 *
 *  struct sname {
 *      char fld1[64];
 *      PADTO(68,sname,pname);
 *  };
 * will make:
 *
 *  struct sname {
 *      char fld1[64];
 *      UBYTE pname[1];
 *      UBYTE __pname[3];
 *  };
 */


#define PADTO(sz,struc,padfld) \
    UBYTE padfld[1];UBYTE __##padfld[(sz)-OFFSET(struct struc,padfld)-1]

here is FHEAD_COMMON

#define FHEAD_COMMON \
CHUNKID_FIELDS;\
USHORT frame_count;\
USHORT width;\
USHORT height;\
USHORT bits_a_pixel;\
SHORT flags;\
LONG speed;\
USHORT unused;\
Fli_id id;\
USHORT aspect_dx;\
USHORT aspect_dy;\
UBYTE commonpad[38]  /* should be total of 80 bytes (48 for unique) */

and flihead

typedef struct fli_head {
FHEAD_COMMON;
LONG frame1_oset;
LONG frame2_oset;
UBYTE padfill[40];
} Fli_head;

this is Autodesk animator pro. what I am working on is the "reference" implementation for the FLI file format- which you can see a spec for here:

http://www.compuphase.com/flic.htm

Incidentally, I'm pretty sure that what the /source code/ there refers to as "flx" is actually what that webpage there calls "flc" , not what it calls "flx" update: better source for format info http://drdobbs.com/architecture-and-design/184408954

edited to add more information for @JoachimPileborg and others :) — Breton, Nov 12 '11 at 05:58
If you wanted to make all your member accesses that much harder to read and write, you could use `typedef union { struct flx_head { ... } x; Fli_head padding; } Flx_head;`. To make access easier, you could always cast a `Flx_head *` to a `struct flx_head *` and not have an extra `x.` in every member access. (Also, `OFFSET` should be replaced with the standard `offsetof` found in `stddef.h` I think.) — Chris Lutz, Nov 12 '11 at 06:39

score 3 · Answer 1 · answered Nov 12 '11 at 06:18

It isn't pretty, but one possibility is to define another identical structure and use its size to determine the padding for the one you actually want to use:

#define FLX_HEAD \
FHEAD_COMMON;\
LONG frames_in_table; /* size of index */ \
LONG index_oset;    /* offset to index */ \
LONG path_oset     /* offset to flipath record chunk */

struct flx_head_unpadded {
  FLX_HEAD;
};

typedef struct flx_head {
  FLX_HEAD;
  char __flxpad[sizeof(Fli_head)-sizeof(struct flx_head_unpadded)];
} Flx_head;

score 1 · Answer 2 · answered Nov 12 '11 at 05:47

1

I suppose the answer depends on what you're trying to achieve. In most cases, the correct, modern way to pad a struct is not to. The only situation I can think of where it's legitimate to pad a struct is when you have a library interface where the caller creates objects of a structure type and passes pointers to them to the library, and where you want to leave room to add additional fields to the structure without breaking the ABI. In this case, I would start out with something like char pad[256]; and change it to char pad[256-3*sizeof(long)]; or similar as you add fields (making sure to avoid internal padding when you add fields).

answered Nov 12 '11 at 05:47

R.. GitHub STOP HELPING ICE

208,859
35
376
711

I'm not specifically trying to do anything other than make this code compile. What the program implements is a particular file format which must adhere to specific offsets to be broadly compatible, so arbitrarily padding it won't cut it, and I have no need to add extra fields since it's a 30 year old format. – Breton Nov 12 '11 at 06:00
Structures have nothing to do with file formats. If you think they do, you're doing something very wrong. – R.. GitHub STOP HELPING ICE Nov 12 '11 at 13:06
You write serialize and deserialize functions to write or read the bytestream. – R.. GitHub STOP HELPING ICE Nov 13 '11 at 14:36
interesting, tell me more. Do you have links to code examples? – Breton Nov 14 '11 at 03:09

score 1 · Answer 3 · answered Nov 12 '11 at 10:05

Define it in a union with a byte/char array of the desired size?

I can think quickly of some scenarios where this is needed:

1) Compatibility with old software that uses flat binary files to store data, (as in OP).

2) Interaction with drivers and/or hardware

3) Forcing structs to be an exact multiple of the cache line size to prevent false sharing in inter-thread comms.

score 0 · Answer 4 · answered Nov 12 '11 at 16:55

If you only want to achieve specific size you can use (for sure working in GCC):

typedef union {
struct {
    FHEAD_COMMON;
    LONG frames_in_table; /* size of index */
    LONG index_oset;    /* offset to index */
    LONG path_oset;     /* offset to flipath record chunk */
};
uint8_t __padding[128];
} Flx_head;

void test() {
    Flx_head boo;
    boo.frames_in_table= 0;
}

I am not sure if this is modern enough. If youc compiler does not support anonymous struct (the one inside union) it will get "messy".

Also you must remember that struct is now padded, but not aligned to specific data size.

score -1 · Answer 5 · answered Nov 12 '11 at 06:37

-1

thanks everyone. not sure who to award the green checkmark to, since I found this solution as a result of everyone kind of hinting and pointing in the right direction. After looking at the problem, it struck me that the struct just needs to be exactly 128 bytes. I "hand parsed" the macro, cross referencing with the spec and ended up with this:

typedef struct flx_head {
FHEAD_COMMON;
LONG frames_in_table; /* size of index */
LONG index_oset;    /* offset to index */
LONG path_oset;     /* offset to flipath record chunk */

    UBYTE flxpad[36];
} Flx_head;

which is 128-(80+4+4+4) = 36

answered Nov 12 '11 at 06:37

Breton

15,401
3
59
76

But this is basically @Vaughn's solution, no? only that his is more portable. – Jens Gustedt Nov 12 '11 at 08:56
@Vaughn's solution involves a redundant copy of the struct. – Breton Nov 12 '11 at 14:57
in his solution this is only a type declaration, no run time penalty at all. But instead of precalculating the `36` his version is stable if something is added to `FHEAD_COMMON` or if the actual compiler decides to do the padding of that part differently. – Jens Gustedt Nov 12 '11 at 15:18
If something is added to FHEAD_COMMON, or the compiler decides to pad that part differently, then the program is broken. :(. I have found that I need to use #pragma pack(push, 1) to get these structs to work right. – Breton Nov 12 '11 at 15:26

What's the correct (modern) way to pad a struct?

5 Answers5