How to repeat a block of C code, every time just changing a few keywords

Question

I'd like to repeat a block of C code N times, every time just changing just a few words (like variable names, variable types...).

I'd like to obtain a final effect similar to NumPy's preprocessor for .src files. For example in their code you can see that this:

/**begin repeat
 * #name = number, integer, signedinteger, unsignedinteger, inexact,
 *         floating, complexfloating, flexible, character#
 * #NAME = Number, Integer, SignedInteger, UnsignedInteger, Inexact,
 *         Floating, ComplexFloating, Flexible, Character#
 */
NPY_NO_EXPORT PyTypeObject Py@NAME@ArrType_Type = {
    PyVarObject_HEAD_INIT(NULL, 0)
    .tp_name = "numpy.@name@",
    .tp_basicsize = sizeof(PyObject)
};
/**end repeat**/

would be turned into something like this:

NPY_NO_EXPORT PyTypeObject PyNumberArrType_Type = {
    PyVarObject_HEAD_INIT(NULL, 0)
    .tp_name = "numpy.number",
    .tp_basicsize = sizeof(PyObject)
};
NPY_NO_EXPORT PyTypeObject PyIntegerArrType_Type = {
    PyVarObject_HEAD_INIT(NULL, 0)
    .tp_name = "numpy.integer",
    .tp_basicsize = sizeof(PyObject)
};
NPY_NO_EXPORT PyTypeObject PySignedIntegerArrType_Type = {
    PyVarObject_HEAD_INIT(NULL, 0)
    .tp_name = "numpy.signedinteger",
    .tp_basicsize = sizeof(PyObject)
};
// And so on for all the couples of `name` and `NAME`.

Is there a way to do the same thing, but without having to use an external preprocessor? I thought about using macros but I don't think that's the case.

The "trivial" solution is to create a template, fill in the things that change, and just generate source code. — Dave Newton, Apr 07 '21 at 15:39
@DaveN ...which I think is basically what Numpy does and is apparently unsuitable because it's "too hackish". (But I'm not sure I see any other options that aren't largely variations of that) — DavidW, Apr 07 '21 at 15:45
Ah; I didn't look at the Numpy code, just the hint in the OP's question. I don't know if I consider it "hackish" or not--if you have to explicitly generate a bunch of C code short of doing even *more* horrible things at the binary level... — Dave Newton, Apr 07 '21 at 15:56
"Is there a way to solve my problem" What exactly *is* your problem that needs solving? Are you aware that Python ships with various tools to interact/use raw c data, such as ``ctypes``, ``struct`` and ``array``? Are you aware of tooling to simplify writing C extensions/wrappers, such as Cython? — MisterMiyagi, Apr 07 '21 at 17:12
@MisterMiyagi yeah, i know a bit of Cython and i also have used it in other projects, but the problem, also using Cython, persists. — Giuppox, Apr 07 '21 at 19:22
If I understand your question correctly, it boils down to "how to repeat code for multiple types in C". It does not appear to be specific to the Python C API, or even Python itself. Have you considered C macros or C++ templates? — MisterMiyagi, Apr 07 '21 at 19:58
@MisterMiyagi I thought about using C macros... but i couldn't find any "repeating-code" information related to them. — Giuppox, Apr 07 '21 at 20:02
So instead of typing ``macro(int, INT)``, ``macro(double, Double)`` and so on for each type you want a single ``all_macro(int, INT, double, DOUBLE, ...)`` and so on for each type? — MisterMiyagi, Apr 07 '21 at 20:09
@MisterMiyagi yeah, what i want is repeating a block of C code `N` times, every time just changing just a few words (like var names or types...) — Giuppox, Apr 07 '21 at 20:14
First: Ask exactly 1 question. PS Put what is needed to ask your question in your post. Do not give links & expect us to read them or know what bits are relevant or how they are relevant. You should be doing that & forming your question from it. PS You never even say what "create wrappers for C fixed data types" means exactly. PS Please clarify via edits, not comments. PS Things like "something equivalent to" are not helpful. You don't clearly say what is (not) an example of. — philipxy, Apr 07 '21 at 23:25

David Ranieri · Accepted Answer · 2021-04-09T19:45:55.280

Does an X-macro helps?

#define LIST_OF_TYPES               \
    X(Number, number)               \
    X(Integer, integer)             \
    X(SignedInteger, signedinteger)

#define X(type, name)                                       \
    NPY_NO_EXPORT PyTypeObject Py##type##ArrType_Type = {   \
        PyVarObject_HEAD_INIT(NULL, 0)                      \
        .tp_name = "numpy."#name,                           \
        .tp_basicsize = sizeof(PyObject)                    \
    };
LIST_OF_TYPES
#undef X

or pass the second argument between quotes:

#define LIST_OF_TYPES                   \
    X(Number, "number")                 \
    X(Integer, "integer")               \
    X(SignedInteger, "signedinteger")

#define X(type, name)                                       \
    NPY_NO_EXPORT PyTypeObject Py##type##ArrType_Type = {   \
        PyVarObject_HEAD_INIT(NULL, 0)                      \
        .tp_name = "numpy."name,                            \
        .tp_basicsize = sizeof(PyObject)                    \
    };
LIST_OF_TYPES
#undef X

or if you prefer, a function-like macro, in this case I will move the semicolon to the macro call:

#define PY_TYPE_OBJECT(type, name)                          \
    NPY_NO_EXPORT PyTypeObject Py##type##ArrType_Type = {   \
        PyVarObject_HEAD_INIT(NULL, 0)                      \
        .tp_name = "numpy."name,                            \
        .tp_basicsize = sizeof(PyObject)                    \
    }

PY_TYPE_OBJECT(Number, "number");
PY_TYPE_OBJECT(Integer, "integer");
PY_TYPE_OBJECT(SignedInteger, "signedinteger");

Hi, thanks for answering. I don't really understand the use of `#`s in your macro definitions (like in `Py##type##ArrType_Type `). — Giuppox, Apr 09 '21 at 19:20
A single `#` is used to stringify, while `##` is used to concatenate, you can see a useful example of both tokens here: https://gcc.gnu.org/onlinedocs/cpp/Concatenation.html — David Ranieri, Apr 09 '21 at 19:30
As for your last example, is there a way to do the exact same thing but defining a function? — Giuppox, Apr 09 '21 at 21:04
@Giuppox I don't think so, you can do some tricks with `enum`s but you would end up assigning a stucture/variable at local scope (the scope of the function) and I don't think that's what you want. — David Ranieri, Apr 09 '21 at 21:14
Of course you can return an allocated object from a function using `PyObject *object = PyObject_New(...)` as described in the [documentation](https://docs.python.org/3/c-api/allocation.html), but that implies using the heap. — David Ranieri, Apr 09 '21 at 21:51

mkayaalp · Answer 2 · 2021-04-09T21:05:19.060

You can write a preprocessor macro:

#define PYOBJ(t, n) \
NPY_NO_EXPORT PyTypeObject Py ## t ## ArrType_Type = { /*
*/    PyVarObject_HEAD_INIT(NULL, 0) /*
*/    .tp_name = "numpy." # n, /*
*/    .tp_basicsize = sizeof(PyObject), /*
*/}

PYOBJ(Number, number);
PYOBJ(Integer, integer);
PYOBJ(SignedInteger, signedinteger);

Running gcc -E -CC test.c produces this:

NPY_NO_EXPORT PyTypeObject PyNumberArrType_Type = { /*
*/ PyVarObject_HEAD_INIT(NULL, 0) /*
*/ .tp_name = "numpy." "number", /*
*/ .tp_basicsize = sizeof(PyObject), /*
*/};
# 9 "test.c"
NPY_NO_EXPORT PyTypeObject PyIntegerArrType_Type = { /*
*/ PyVarObject_HEAD_INIT(NULL, 0) /*
*/ .tp_name = "numpy." "integer", /*
*/ .tp_basicsize = sizeof(PyObject), /*
*/};
# 10 "test.c"
NPY_NO_EXPORT PyTypeObject PySignedIntegerArrType_Type = { /*
*/ PyVarObject_HEAD_INIT(NULL, 0) /*
*/ .tp_name = "numpy." "signedinteger", /*
*/ .tp_basicsize = sizeof(PyObject), /*
*/};

See the preprocessor macro documentation on stringizing:

When a macro parameter is used with a leading ‘#’, the preprocessor replaces it with the literal text of the actual argument, converted to a string constant.

and on concatenation:

The ‘##’ preprocessing operator performs token pasting. When a macro is expanded, the two tokens on either side of each ‘##’ operator are combined into a single token, which then replaces the ‘##’ and the two original tokens in the macro expansion.

You need to have two parameters to the macro, even though the second one is the lowercase version of the first one (Integer vs. integer). Because the preprocessor does not allow token transformations. See the question Can you capitalize a pasted token in a macro?.

As to the /* */ comments and the -CC compiler flag I used, it puts newlines in the preprocessor output to make it readable. The idea comes from this question: How to make G++ preprocessor output a newline in a macro?. You could instead write:

#define PYOBJ(t, n) \
NPY_NO_EXPORT PyTypeObject Py ## t ## ArrType_Type = { \
    PyVarObject_HEAD_INIT(NULL, 0) \
    .tp_name = "numpy." # n, \
    .tp_basicsize = sizeof(PyObject), \
}

The `/* */` is a really nice thought. If i understand correctly someone might want to just remove the comments and put a backslash, and the code would work in exact same way. — Giuppox, Apr 09 '21 at 21:02

How to repeat a block of C code, every time just changing a few keywords

2 Answers2