23

I have a Python project with many sub-modules that I package up with distutils. I would like to build some Python extensions in C to live in some of these sub-modules but I don't understand how to get the Python extension to live in a submodule. What follows is the simplest example of what I'm looking for:

Here is my Python extension c_extension.c:

#include <Python.h>

static PyObject *
get_answer(PyObject *self, PyObject *args)
{
    return Py_BuildValue("i", 42);
}

static PyMethodDef Methods[] = {
    {"get_answer",  get_answer, METH_VARARGS, "The meaning of life."},
    {NULL, NULL, 0, NULL}
};

PyMODINIT_FUNC
initc_extension(void) {
  (void) Py_InitModule("c_extension", Methods);
}

And here is a setup.py that works:

from distutils.core import setup
from distutils.extension import Extension

setup(name='c_extension_demo',
      ext_modules = [Extension('c_extension', sources = ['c_extension.c'])])

After installing in an virtualenv I can do this:

>>> import c_extension
>>> c_extension.get_answer()
42

But I would like to have c_extension live in a sub-module, say foo.bar. What do I need to change in this pipeline to be able to get the behavior in the Python shell to be like this:

>>> import foo.bar.c_extension
>>> foo.bar.c_extension.get_answer()
42
Rich
  • 12,068
  • 9
  • 62
  • 94
  • Related: https://stackoverflow.com/q/12097755/7954504 – Brad Solomon Oct 08 '19 at 15:32
  • For `python >= 3.10` , `distutils` is deprecated, please see [my answer in another thread](https://stackoverflow.com/a/67759085/9853105) for detail – Ham May 30 '21 at 08:21

1 Answers1

17

Just change

Extension('c_extension', ...)

to

Extension('foo.bar.c_extension', ...)

You will need __init__.py files in each of the foo and bar directories, as usual. To have these packaged with the module in your setup.py, you need to add

packages = ['foo', 'foo.bar'],

to your setup() call, and you will need the directory structure

setup.py
foo/
    __init__.py
    bar/
        __init__.py

in your source directory.

nneonneo
  • 171,345
  • 36
  • 312
  • 383
  • Thanks for the answer but that didn't work for me. I made `foo/bar` directory with an `__init__.py` in each plus added the `foo.bar.` prefix in the setup command but in the python shell this happens: >>> import foo.bar.c_extension Traceback (most recent call last): File "", line 1, in ImportError: No module named c_extension – Rich Aug 23 '12 at 18:35
  • What version of Python are you using? I'm testing on 2.7.3. – nneonneo Aug 23 '12 at 18:37
  • 5
    Don't test from your source directory. That will try to import from the foo/bar in the source dir which doesn't have c_extension. – nneonneo Aug 23 '12 at 18:39
  • Oh RIGHT. That was it. Thank you for the help! – Rich Aug 23 '12 at 18:41