How to shorten these lines using preprocessor?

Question

I have these lines in my code and was thinking that there may be some nice way to use preprocessor to generate these lines (0 through 31).

    Mem_type MEM_0[MAX_NUM_MEM];
    Mem_type MEM_1[MAX_NUM_MEM];
    Mem_type MEM_2[MAX_NUM_MEM];
    Mem_type MEM_3[MAX_NUM_MEM];
    Mem_type MEM_4[MAX_NUM_MEM];
    Mem_type MEM_5[MAX_NUM_MEM];
    ...
    Mem_type MEM_30[MAX_NUM_MEM];
    Mem_type MEM_31[MAX_NUM_MEM];

    Mem_type *MEM[NUM_BANKS];

    MEM[0] = MEM_0;
    MEM[1] = MEM_1;
    MEM[2] = MEM_2;
    MEM[3] = MEM_3;
    MEM[4] = MEM_4;
    MEM[5] = MEM_5;
    ...
    MEM[30] = MEM_30;
    MEM[31] = MEM_31;

For example, something like:

    #define Gen(n) MEM[n] = MEM_n
    #for (k=0; k<32; k++) Gen(k);

(The reason why I don't do like the below is that I found that my machine has some maximum contiguous array size limit, so I tried to split it into separate arrays so that I can have larger aggregated MEM size.)

    Mem_type MEM[NUM_BANKS][MAX_NUM_MEM];

@tkausl That's what I was doing originally. But I figured that my machine seems to have some maximum contiguous array size limit, so I decided to split the 'MEM' and scatter so I can have larger aggregated MEM size. — Kay K., Apr 16 '19 at 07:02
The boost preprocessor library can do that sort of thing, though it would likely be even uglier to decode for someone unfamiliar with the code. — SoronelHaetir, Apr 16 '19 at 07:11
Why do you need a preprocessor for that? Why not use a normal loop? Or template metaprogramming? — Kit., Apr 16 '19 at 07:14
And by the way, what is your machine and what is the value of MAX_NUM_MEM? — Kit., Apr 16 '19 at 07:16
@SoronelHaetir Thanks. I'm not familiar with boost preprocessor but I'll try it. — Kay K., Apr 16 '19 at 07:18
@Kit. Thanks. I haven't heard of metaprogramming but I'll try it. The MAX_NUM_MEM*NUM_BANKS is not that large (about 128K), but the Mem_type (a struct) is about 140 Bytes. If I do more than 128K then I get memory error. I'm on a Linux server. — Kay K., Apr 16 '19 at 07:23
Instead of using the stack, you could try to allocate the array with `new`. The heap has a lot more space. Alternatively you could try using not arrays but something like vectors. If the array size is still a problem (which it shouldn't according to https://stackoverflow.com/questions/216259/is-there-a-max-array-length-limit-in-c ), then you can push new, smaller arrays to the vector programatically instead of having one large one. — FalcoGer, Apr 16 '19 at 07:36
@FalcoGer Thanks for all your suggestions. Although I think I am fine with the above one (except I wished to shorten it), I'll follow what you told me for the future. — Kay K., Apr 16 '19 at 07:47
I know of no Linux machine that would have a contiguous array size limit like this. Mos likely, you are just running out of memory (stack size?), and just splitting the array into multiple ones won't help you. — Kit., Apr 16 '19 at 08:19

score 3 · Answer 1 · answered Apr 16 '19 at 07:28

There are articles on the net about symbolic computations on preprocessing stage, one typical instance would be http://jhnet.co.uk/articles/cpp_magic

If the machinery defined there is too much for you and you don't care much about the prettiness of generated code you could use a cheap alternative, something like (untested):

#define ONE_MEM(i, a) MemType mem_ ## a[MAX_MEM]; mem[i] = mem_ ## a
#define MEM_1(i, a) ONE_MEM(i, a); ONE_MEM(i + 1, a ## 1)
#define MEM_2(i, a) MEM_1(i, a); MEM_1(i + 2, a##2)
#define MEM_4(i, a) MEM_2(i, a); MEM_2(i + 4, a##4)

and so forth, now it's logarithmic in amount of macros defined.

(Haven't tested it, the actual definition might need a concat indirection or two.)

There can be improvements, like declaring a macro parameter to substitute for mem etc.

Thanks. It's an interesting idea to use power of 2. I don't care actually numbering in the variable names, so I think this may work for me. — Kay K., Apr 16 '19 at 07:32

score 3 · Accepted Answer · answered Apr 16 '19 at 07:39

Using boost/preprocessor/repetition/repeat.hpp :

#include <boost/preprocessor/repetition/repeat.hpp>
class Mem_type {};

#define MAX_NUM_MEM  5
#define NUM_BANKS    5

#define MEM_DECL(z, n, text)  Mem_type MEM_## n[MAX_NUM_MEM];
#define MEM_MEMB(z, n, text)  MEM_## n,

// expands to `Mem_type MEM_?[MAX_NUM_MEM];`
BOOST_PP_REPEAT(NUM_BANKS, MEM_DECL, ())

Mem_type *MEM[NUM_BANKS] = {
    // expands to `MEM_?,`
    BOOST_PP_REPEAT(NUM_BANKS, MEM_MEMB, ())
};

Thanks. This is sort of what I was looking for. – Kay K. Apr 16 '19 at 07:44 — Kay K., Apr 16 '19 at 07:44

score 2 · Answer 3 · edited Jan 01 '20 at 15:53

2

The stack is very limited and should not be used to allocate huge data structures like this as discussed here. Instead try to allocate your memory using new[]. If you do need multidimensional arrays you can use an array of pointers that point to arrays pointing to your structures as described here.

However as your initial intention was to have a single array, this should do the trick without the need of the preprocessor:

Mem_type* MEM = new Mem_type[MAX_NUM_MEM]; // MAX_NUM_MEM is multiplied by NUM_BANKS here
// do things [...]
delete[] MEM;
MEM = nullptr;

It would be an idea to wrap this up in a class, allocate in the constructor, throw an exception if allocation fails and deallocate in the destructor.

edited Jan 01 '20 at 15:53

phuclv

37,963
15
156
475

answered Apr 16 '19 at 07:50

FalcoGer

2,278
1
12
34

1

Thanks. I'll try it. One question is - can 'new' return a null pointer (allocation failure)? I'm asking because of the 'if' statement. (BTW, although I think this answer actually helps me better, I feel I need to accept the other answer, as that is answering the original question on preprocessor. Sorry about that.) – Kay K. Apr 16 '19 at 08:04
According to http://www.cplusplus.com/reference/new/operator%20new[]/ new throws an exception, unless you specify otherwise. It used to return a null pointer on failure in older c++ standards, if I remember correctly. – FalcoGer Apr 16 '19 at 08:10
there's no `new` keyword in C – phuclv Jan 02 '20 at 01:39

score 1 · Answer 4 · edited Apr 16 '19 at 08:30

1

Using dynamic allocation with STL vectors:

#include <vector>

class Mem_type {};
const int MAX_NUM_MEM = 5;
const int NUM_BANKS   = 5;

// allocates NUM_BANKS vectors with MAX_NUM_MEM objects of Mem_type
std::vector<std::vector<Mem_type>> MEM(NUM_BANKS, std::vector<Mem_type>(MAX_NUM_MEM));

edited Apr 16 '19 at 08:30

FalcoGer

2,278
1
12
34

answered Apr 16 '19 at 08:13

KamilCuk

120,984
8
59
111

score 1 · Answer 5 · answered Apr 16 '19 at 08:33

You can use an X Macro along with token concatenating

#define LIST_OF_MEMS    \
X(0)                    \   
X(1)                    \
X(2)                    \
X(3)                    \
X(4)                    \
X(5)                    \
...                     \
X(30)                   \
X(31)                   \

Now you can use it every time you want to do anything with all the MEMs. Note that using all caps for a variable name is a bad idea

// declaration
#define X(num) Mem_type mem_##num[MAX_NUM_MEM];
LIST_OF_MEMS
#undef X

// assignment
#define X(num) MEM[num] = mem_##num;
LIST_OF_MEMS
#undef X

How to shorten these lines using preprocessor?

5 Answers5