Building compile-time arbitrary length arrays in program memory for AVR

Question

I'm trying to figure out a smart-ass way to build a composite compile-time array for the AVR architecture. The array should be structured as follows:

• it should reside entirely in program memory;
• it is comprised of a continuous series of (unsigned) bytes, aka uint8_t;
• it should be built using segments of bytes of arbitrary length;
• a segment is comprised, in order, of a length byte and a series of data bytes, with the length byte being the number of data bytes.

Here's an example of such an array:

static const uint8_t data[] PROGMEM = {
    1, 0x01,
    3, 0xBE, 0x02, 0x00,
    3, 0x3D, 0x33, 0x33,
    15, 0xE1, 0xD0, 0x00, 0x05, 0x0D, 0x0C, 0x06, 0x2D, 0x44, 0x40, 0x0E, 0x1C, 0x18, 0x16, 0x19,
    0 /* end of the sequence */
};

I want to avoid the burden of adjusting the length byte everytime I'd add or remove a byte from the sequence, for instance, in some form of pseudo code:

BEGINNING_OF_THE_SEQUENCE(identifier)
    SEGMENT(0x01),
    SEGMENT(0xBE, 0x02, 0x00),
    ...
END_OF_THE_SEQUENCE()

In the above example I chose an explicit byte array declaration but it could be built in any way, for example using a structure, whatsoever. The only prerequisite is that the order of appearance must be guaranteed.

So in short I'd like to "concatenate" series of bytes, which length must be calculated at compile-time and put in front of each byte series as the length byte of the series itself.

I've considered using variadic macros but I'd also like to investigate other means, such as class and function templates, meta-programming, whatnot, with the smallest code in mind. I'd also like not to resort to C++11-specifics as the support with the current avr-gcc compiler I'm using is limited so far.

I've got a hunch it is possible using templates but I'm stuck. Any ideas?

Answer 1

Here's a simple example for C++11 and beyond that might help:

template <typename ...Args>
constexpr std::size_t n_args(Args...) { return sizeof...(Args); }

#define ELEM(...) n_args(__VA_ARGS__), __VA_ARGS__

#include <iostream>

int main()
{
    unsigned int a[] = { ELEM(4, 9, 16),
                         ELEM(100),
                         ELEM(10, 20, 30, 40, 50),
    };

    for (auto n : a ) std::cout << n << " ";
    std::cout << '\n';
}

Alternatively, you could use sizeof a compound char-array literal in place of n_args, that's if you want a C99 solution rather than a C++11 one:

#define ELEM(...) sizeof((char[]){__VA_ARGS__}), __VA_ARGS__

I'm not aware of a similarly simple approach that would work in C++03.

Answer 2

Here is one approach:

#include <stdint.h>
#include <stdio.h>
#define PROGMEM
#define _ARGSIZE(...) sizeof((uint8_t[]){__VA_ARGS__})/sizeof(uint8_t)
#define _SEGMENT(...)  _ARGSIZE( __VA_ARGS__ ), __VA_ARGS__
#define BEGINNING_OF_THE_SEQUENCE(__id)   uint8_t static const __id[] PROGMEM =  {
#define END_OF_THE_SEQUENCE() }

BEGINNING_OF_THE_SEQUENCE(data)
    _SEGMENT(0x01),
    _SEGMENT(0xBE, 0x02, 0x00),
    _SEGMENT(0xDE, 0xAD, 0xBE, 0xEF)
END_OF_THE_SEQUENCE();

int main() {
    int k, counter = data[0];
    for (k = 0; k < sizeof(data); k++) {
        fprintf(stderr, "%02x ", data[k]);
        if(counter-- == 0) {
            counter = data[1+k];
            fprintf(stderr, "\n");
        }
    }
}

This approach is C99 compatible.

The macros above can be modified with some care to take care of any type of data structure passed instead of uint8_t (including :P structures within structures)

Answer 3

Macro-free C++11 approach (v2):

#include <array>
#include <iostream>
#include <cstddef>
#include <cstdint>

//  This template will be instantiated repeatedly with VItems list
//  populated with new items.
template<typename TItem,  TItem... VItems> class
t_PackImpl
{
    //  This template will be selected for second and all other blocks.
    public: template<TItem... VInnerItems> using
    t_Pack = t_PackImpl
    <
        TItem
    //  add all previous items
    ,   VItems...
    //  add item holding amount of items in new block
    ,   TItem{static_cast<TItem>(sizeof...(VInnerItems))}
    //  add new block items
    ,   VInnerItems...
    >;

    //  This method will be called on the last instantiated
    //  template with VItems containing all the items.
    //  Returns array containing all the items with extra 0 item at the end.
    public: static constexpr auto
    to_array(void) -> ::std::array<TItem, sizeof...(VItems) + ::std::size_t{1}>
    {
        return {VItems..., TItem{}};
    }
};

//  This template will be instantiated just once.
//  Starts t_PackImpl instantiation chain.
template<typename TItem> class
t_BeginPack
{
    //  This template will be selected for first block.
    public: template<TItem... VInnerItems> using
    t_Pack = t_PackImpl
    <
        TItem
    //  add item holding amount of items in new block
    ,   TItem{static_cast<TItem>(sizeof...(VInnerItems))}
    //  add new block items
    ,   VInnerItems...
    >;
};

int main()
{
    {
        constexpr auto items
        {
            t_BeginPack<::std::uint8_t>::t_Pack<42>::to_array()
        };
        for(auto const & item: items)
        {
            ::std::cout << static_cast<::std::uint32_t>(item) << ::std::endl;
        }
    }
    ::std::cout << "----------------" << ::std::endl;
    {
        constexpr auto items
        {
            t_BeginPack<::std::uint8_t>::t_Pack<0, 1, 2>::to_array()
        };
        for(auto const & item: items)
        {
            ::std::cout << static_cast<::std::uint32_t>(item) << ::std::endl;
        }
    }
    ::std::cout << "----------------" << ::std::endl;
    {
        constexpr auto items
        {
            t_BeginPack<::std::uint8_t>::
                t_Pack<0, 1, 2>::
                t_Pack<0, 1>::
                t_Pack<0, 1, 2, 3, 4, 5>::to_array()
        };
        for(auto const & item: items)
        {
            ::std::cout << static_cast<::std::uint32_t>(item) << ::std::endl;
        }
    }
    return(0);
}

Run online