10

I'm working on some C++ code for an embedded system. The I/O interface the code uses requires that the size of each message (in bytes) is a power of two. Right now, the code does something like this (in several places):

#pragma pack(1)
struct Message
{
   struct internal_
   {
      unsigned long member1;
      unsigned long member2;
      unsigned long member3;
      /* more members */
   } internal;
   char pad[64-sizeof(internal_)];
};
#pragma pack()

I'm trying to compile the code on a 64-bit Fedora for the first time, where long is 64-bits. In this case, sizeof(internal_) is greater than 64, the array size expression underflows, and the compiler complains that the array is too large.

Ideally, I'd like to be able to write a macro that will take the size of the structure and evaluate at compile time the required size of the padding array in order to round the size of the structure out to a power of two.

I've looked at the Bit Twiddling Hacks page, but I don't know if any of the techniques there can really be implemented in a macro to be evaluated at compile time.

Any other solutions to this problem? Or should I perpetuate the problem and just change the magical 64 to a magical 128?

Nick Meyer
  • 39,212
  • 14
  • 67
  • 75

12 Answers12

15

Use a template metaprogram. (Edited in response to comment).

#include <iostream>
#include <ostream>
using namespace std;

template <int N>
struct P
{
    enum { val = P<N/2>::val * 2 };
};

template <>
struct P<0>
{
    enum { val = 1 };
};

template <class T>
struct PadSize
{
    enum { val = P<sizeof (T) - 1>::val - sizeof (T) }; 
};

template <class T, int N>
struct PossiblyPadded
{
    T       payload;
    char    pad[N]; 
};

template <class T>
struct PossiblyPadded<T, 0>
{
    T       payload;
};

template <class T>
struct Holder : public PossiblyPadded<T, PadSize<T>::val>
{
};


int main()
{
    typedef char Arr[6];

    Holder<Arr> holder;
    cout << sizeof holder.payload << endl;

    // Next line fails to compile if sizeof (Arr) is a power of 2
    // but holder.payload always exists
    cout << sizeof holder.pad << endl;
}
fizzer
  • 13,551
  • 9
  • 39
  • 61
  • I like this, but it fails when the contents are an even power of 2. The declaration of pad gets error C2466: cannot allocate an array of constant size 0 – Drew Hoskins Aug 06 '09 at 16:39
  • 1
    I suppose you could derive the PossiblyPadded from T instead of having a T member (for suitable T), and save yourself the annoying member access syntax. – fizzer Aug 06 '09 at 17:18
  • 1
    @fizzer, is that guarenteed to not add padding? I'd use a union myself. And you can always make a T *operator->()... – bdonlan Aug 06 '09 at 18:41
  • Thanks! I like bdonlan's answer too, but I think this is probably a better general solution. – Nick Meyer Aug 06 '09 at 21:53
  • 1
    Love it! I found this while looking for a template metaprogramming way of getting the next power of two at compile time. Works great! – Mark Borgerding Jan 20 '11 at 16:26
6

Why not use a union?

union Message
{
    struct internal_
    {
        unsigned long member1;
        /* more members */
    };
    char[64];
};

or better yet use anonymous structs

union Message
{
    struct
    {
        unsigned long member1;
        /* more members */
    };
    char[64];
};

So you can access members like this: Message.member1;

Edit: obviously this doesn't solve your greater than 64 problem, but provides a cleaner way of padding.

Niki Yoshiuchi
  • 16,883
  • 1
  • 35
  • 44
  • Actually this is dangerous, since member1 and member2 will NOT be aligned at the same address on different platforms. Since this is IO code, this would result in errors. – Dominik Weber Sep 28 '10 at 20:16
6

Probably the most obvious way would be to just use the ternary operator:

#define LOG2_CONST(n) ((n) <= 1 ? 0 :
                      ((n) <= 2 ? 1 :
                      ((n) <= 4 ? 2 :
                      /* ... */
                      ))))))))))))))))))))))))))))))
#define PADDED_STRUCT(ResultName, BaseName) \
  typedef union { BaseName data; char pad[1 << LOG2_CONST(sizeof(BaseName))]; } ResultName
bdonlan
  • 224,562
  • 31
  • 268
  • 324
5

How about just writing a small wrapper around the send and receive message function that handle any size message and they just allocate a larger buffer ( next power of 2 ) and memclear it, copy the struct to the beginning and send it along.

KPexEA
  • 16,560
  • 16
  • 61
  • 78
4

One way of working around the problem would be to replace the hardcoded 64 with a multiple of size(long), turning the padding into something like this:

char pad[4*sizeof(unsigned long) - sizeof(internal_)];

It's ugly but it should be portable to 64-bit.

That said, an API that requires the message size to be a power of 2 sounds a bit odd and like a design issue. Requiring the size being an even number makes sense as on some processors to pay quite a penalty for accessing data on odd addresses but your #pragma pack almost makes that inevitable.

Timo Geusch
  • 24,095
  • 5
  • 52
  • 70
2

You're already using #pragma pack, I don't know what compiler(s) your using specifically, but you should see if they support arguments for pack that control the alignment/padding and then you can just get rid of the padding field altogether. I know MSVC's version of pragma pack supports this, as does GCC's.

Logan Capaldo
  • 39,555
  • 5
  • 63
  • 78
2

You can macroize this as follows (for 32-bit architecture):

#define align_step(N, shift) ((N) | ((N) >> shift))
#define align_up(N) (align_step(align_step(align_step(align_step(align_step((N)-1, 1), 2), 4), 8), 16) + 1)
#define alignment_padding(N) (align_up((N)) - (N))

Then you can apply that using the union trick or some other means. In your example:

#pragma pack(1)
struct Message
{
   struct internal_
   {
      unsigned long member1;
      unsigned long member2;
      unsigned long member3;
      /* more members */
   } internal;
   char pad[alignment_padding(sizeof(internal_))];
};
#pragma pack()
MSN
  • 53,214
  • 7
  • 75
  • 105
1

You can get a compile-time constant for the size of the structure rounded up to a power of two using templates :

template<int N, int C = 1>
struct Recurse
{
    enum {result = Recurse<N/2, C*2>::result};
};

template<int C>
struct Recurse<0, C>
{
    enum {result = C};
};

template<typename T>
struct Calc
{
    enum {size = Recurse<sizeof(Test)-1>::result};
};

struct Test
{
    int a;
    double b;
    double c;
};

int main()
{
    std::cout << sizeof(Test) << " -> " << Calc<Test>::size << std::endl;
    return 0;
}

The padding value should then be easy.

jon hanson
  • 8,722
  • 2
  • 37
  • 61
0
union Message
{
    struct
    {
        unsigned long member1;
        unsigned long member2; //...
    };
    char pad[1 << 5]; //change 5 to whatever size you need...
};

Would be a little cleaner.

plastic chris
  • 602
  • 6
  • 11
0

I like Niki's answer, especially the part with anonymous structs.

One thing that answer didn't solve was the greater-than-64-bytes problem, but that can be solved by conditionally declaring a char[128] struct member if sizeof(long)==8 and declaring char[64] otherwise.

Community
  • 1
  • 1
azheglov
  • 5,475
  • 1
  • 22
  • 29
0

And yet another template solution (robbing hugely from fizzer):

#include <iostream>
#include <ostream>
using namespace std;

template <int TSize, int PSize = 1, bool = false>
struct PadSize
{
  static const int val =
    ( PadSize <    TSize, (PSize*2), (TSize <= (PSize*2)  )    > :: val );
};

template < int TSize, int PSize>
struct PadSize <TSize, PSize, true>  // As soon as TSize is <= to PSize
{
  static const int val = PSize;
};

int main()
{
    typedef char Arr[8];
    char pad[ PadSize <sizeof(Arr)>::val  ];

    cout << sizeof pad << endl;
}

My approach is simply to keep doubling the padding size until it's at least as big as the type size.

Community
  • 1
  • 1
Richard Corden
  • 21,389
  • 8
  • 58
  • 85
0

Basics

Basing myself on Fizzer's answer, but with a more modern (C++17) style, and what I consider to be a nicer interface, allowing you to pad any struct/class in the following way:

#include "padded.h"

struct UnpaddedStruct
{
    unsigned char unaligned_data[5];
};

using PaddedStruct = Padded<UnpaddedStruct>;

sizeof(UnpaddedStruct) is 5, as expected. sizeof(PaddedStruct) is 8, as wanted.

The biggest difference here is that there's no special data members (aside from possibly padding), nor do you have to write your classes in any particular way to add padding. You simply use the Padded struct for alignment, and otherwise access your struct in the exact same way you would without padding.

Here's a ready-made file you should be able to include as a header to allow for that functionality:

#ifndef PADDED_H
#define PADDED_H
/**
 * padded.h - Padding to align types with powers of two.
 */
#include <cstddef>
#include <cstdint>
#include <limits>

/**
 * pad_size_of<T>() - Get the size required to align T to a power of two.
 * @tparam T: The type to get alignment values for.
 *
 * Return: The amount of padding required to align T to a power of two.
 */
template <typename T>
constexpr size_t pad_size_of()
{
    // Edge case. The highest number which is representable for any size type
    // is going to be one less than the number we want to align it to.
    const size_t max_of = std::numeric_limits<size_t>::max();
    if constexpr(sizeof(T) > max_of / 2) {
            return max_of - sizeof(T) + 1;
    }
    // We want to find the power of two that can fit AT LEAST sizeof(T).
    size_t power_of_two = 1;
    // There's definitely some better way to do this, but this is the most
    // straight forward way to do what it's supposed to do. It's a constexpr
    // function, so there should be no runtime overhead anyway.
    while (power_of_two < sizeof(T)) {
        power_of_two *= 2;
    }
    // Finally, our padding size is simply the difference between the two.
    return power_of_two - sizeof(T);
}

/**
 * struct Padded<T, Size> - A struct including padding for alignment.
 * @tparam T: The class to add padding to.
 * @tparam Size: The amount of padding to add to the class.
 *
 * Simple struct which can be inherited to add padding to any class. Will
 * default to figuring out the amount of padding required to align a class to
 * a power of two, then use that.
 */
template <typename T, size_t Size = pad_size_of<T>()>
struct Padded : public T
{
    /// Simple array of bytes large enough to align the class to powers of two.
    uint8_t padding[Size]{};
};


// Specialization of the above template for the base case, where no padding is
// required for alignment, so none is included.
template <typename T>
struct Padded<T, 0> : public T {};
#endif // PADDED_H

Details

For the especially interested, let's explain what's going on here.

First, I know some people dislike comments/docstrings finding them to be noisy, so let's start with the plain code with no comments:

#include <limits>

template <typename T>
constexpr size_t pad_size_of()
{
    const size_t max_of = std::numeric_limits<size_t>::max();
    if constexpr(sizeof(T) > max_of / 2) {
            return max_of - sizeof(T) + 1;
    }
    size_t power_of_two = 1;
    while (power_of_two < sizeof(T)) {
        power_of_two *= 2;
    }
    return power_of_two - sizeof(T);
}

template <typename T, size_t Size = pad_size_of<T>()>
struct Padded : public T
{
    uint8_t padding[Size]{};
};

template <typename T>
struct Padded<T, 0> : public T {};

The first bit of "magic" here is the constexpr function:

template <typename T>
constexpr size_t pad_size_of()

constexpr functions can be evaluated at compile-time, which means we can use them to figure out, e.g. class properties. We don't have to abuse structs and enums to do whatever, we simply write what we mean.

A constexpr function call may be omitted by the compiler, instead using the result of the function call directly, rather than calling it at runtime. This means there should be no binary bloat or anything, it's just a simple function the compiler can use to figure out how much padding we need.

We then define our Padded struct as a template class:

template <typename T, size_t Size = pad_size_of<T>()>
struct Padded : public T

It takes two template arguments, one being the class you want to add padding to, and the other being the amount of padding you want to add to it. It defaults to figuring that out automatically for you based on the size of the class, with the help of our pad_size_of function, so that's usually not something you have to think about. You can think about it, if you want to e.g. align to a multiple of sizeof(int) rather than a power of two, but power of two should be reasonable in most cases.

Finally, we have a specialization of our Padded struct, namely:

template <typename T>
struct Padded<T, 0> : public T {};

This is the other "magical" bit. If a struct does not require padding, it will not have padding. Simple as. This allows you to e.g. add data members to the underlying struct without having to worry about updating any padding values or whatever. If no overhead is required, there will be no overhead.

Full Example

#include <cstdint>
#include <iostream>
#include "padded.h"

/**
 * struct UnpaddedStruct - Some arbitrary struct with no padding.
 */
struct UnpaddedStruct
{
    /// Whatever unaligned data you want.
    uint8_t data[5];
};

/**
 * struct Struct - An UnpaddedStruct with extra padding.
 */
using Struct = Padded<UnpaddedStruct>;


int main(int argc, char *argv[])
{
    Struct s{'a', 'b', 'c', '\0'};
    std::cout
            << "UNPADDED: " << sizeof(UnpaddedStruct)
            << ", PADDED: " << sizeof(Struct)
            << ", CONTENT: " << s.data
            << std::endl;
    return 0;
}

Output

UNPADDED: 5, PADDED: 8, CONTENT: abc
hodahle
  • 21
  • 3