Compiling Python 3 extension module on Mac

Question

I'm trying to write a Python extension module in C. I'm running macOS Catalina and have a Homebrew installation of Python 3 (with default installation settings). When I try to compile the following file:

#include <Python/Python.h>

static PyObject* world(PyObject* self, PyObject* args)
{
    printf("Hello world!\n");
    Py_RETURN_NONE;
}

static PyMethodDef methods[] = {
    {"world", world, METH_VARARGS, "Prints \"Hello world!\""},
    {NULL, NULL, 0, NULL}
};

static struct PyModuleDef module = {
    PyModuleDef_HEAD_INIT,
    "name for the module",
    "docstring for the module",
    -1,
    methods
};

PyMODINIT_FUNC PyInit_hello(void)
{
    return PyModule_Create(&module);
}

by running gcc hello.c in my terminal, I get the following messages:

hello.c:15:5: error: use of undeclared identifier
      'PyModuleDef_HEAD_INIT'
    PyModuleDef_HEAD_INIT,
    ^
hello.c:14:27: error: variable has incomplete type
      'struct PyModuleDef'
static struct PyModuleDef module = {
                          ^
hello.c:14:15: note: forward declaration of 'struct PyModuleDef'
static struct PyModuleDef module = {
              ^
hello.c:24:12: warning: implicit declaration of function
      'PyModule_Create' is invalid in C99
      [-Wimplicit-function-declaration]
    return PyModule_Create(&module);
           ^
1 warning and 2 errors generated.

Is there a way to fix this? I've tried playing around with the -L and -I flags to no avail. I think this might be happening because it’s using the header for Python 2 rather than Python 3.

score 2 · Accepted Answer · answered Dec 11 '19 at 19:45

2

Use distutils

create setup.py. e.g.

from distutils.core import setup, Extension

def main():
    setup(name="hello",
            version="1.0.0",
            description="desc",
            author="<your name>",
            author_email="a@b.com",
            ext_modules=[Extension("hello",["hello.c"])])

if __name__ == "__main__":
    main()

in c file change include to:

#include <Python.h>

Assuming your c file name is hello.c and setup.py is placed in the same directory:

python3 setup.py install

In your case, verify whether it is "python3" or "python". This should build your module and install it, you should be able to see the compiler and linker commands

answered Dec 11 '19 at 19:45

Icarus3

2,310
14
22

Thanks, I’ll try this when I have the chance. Is this approach new for Python 3? For Python 2, I could just gcc my C file to a .so file and, in the same directory, import the module. – user76284 Dec 11 '19 at 23:47
You still can use gcc but distutils is a standard way of building and distributing modules. It invokes the gcc with the correct flags and link relevant libraries. Distutils has been there since python 1.6 – Icarus3 Dec 12 '19 at 00:57
Do you know why this creates two folders under the created `build` directory? (For me it's `lib.macosx-10.15-x86_64-3.7` which contains `hello.cpython-37m-darwin.so` and `temp.macosx-10.15-x86_64-3.7` which contains `hello.o`.) – user76284 Dec 12 '19 at 01:32
The temp directory holds the object files. Think of it as a temporary work area used by distutil to build the python module. The other one contains the final python module: https://docs.python.org/2.0/inst/how-build-works.html – Icarus3 Dec 12 '19 at 02:12
It seems I can access the module even from a different directory. Is it being installed globally? If so, how can I keep it local for testing purposes, or uninstall it? – user76284 Dec 12 '19 at 02:59
instead of 'install' use 'build' flag. That will just build the module and not install it in site-package. To uninstall you will have to record the path where it installed the module into. That will be "python setup.py install --record files.txt". Then delete the files that are listed in files.txt. https://stackoverflow.com/questions/1550226/python-setup-py-uninstall – Icarus3 Dec 12 '19 at 03:08
I don't know if you're familiar with the Numpy C API, but do you by any chance know why I'm getting the warning *"Using deprecated NumPy API, disable it with #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION"*, even though Homebrew says my numpy is up-to-date (1.17.4)? I'm following the example here: https://scipy-lectures.org/advanced/interfacing_with_c/interfacing_with_c.html#id1 – user76284 Dec 12 '19 at 03:29
Never used it. But the doc says you can ignore it. Visit this link and search for numpy http://docs.cython.org/en/latest/src/userguide/source_files_and_compilation.html#warnings – Icarus3 Dec 12 '19 at 03:35

Compiling Python 3 extension module on Mac

1 Answers1