Making a C++ module part of a Python package

Question

I have the following directory layout

awesome_package
\- module1.py
\- build
   \- module2.so

I currently import module1 as

import awesome_package.module1

and module2 as

import sys
sys.path.append('path/to/awesome_package/build')
import module2

but I would like to be able to use the former syntax.

module2 is created by pybind11 in a fashion like:

PYBIND11_MODULE(module2, module2) {
    module2.doc() = "C++ module wrapped for Python";
    module2.def("some_cpp_function", some_cpp_function) 
}

Then why don't you put C++ module into the same folder as module1? Also python modules written in C/C++ usually have .pyd extension. — , Aug 15 '17 at 15:54
@Ivan Are you saying `import awesome_package.build.module2` would work? — Daniel H, Aug 17 '17 at 19:36
Well, it's not clear from your code if yours is an actual package or not (are there any `__init__.py` files?). But you could use `__init__.py` files and have the the code to append to sys path inside the `__init__.py` file in the build directory. You would need another `__init__.py` file in the root directory, and this one as far as I understand from the way you write can be empty. In that way you wouldn't need every time you write a script to add the code to append to path. — fedepad, Aug 18 '17 at 07:40
Python module written in C/C++ (.pyd, should has Python-specific initialization routine) can be imported just as any other Python module. — , Aug 18 '17 at 09:41
Python modules are normally built with Python's own `distutils`/`setuptools` build system, and you can specify to which package your binary module belongs in `setup.py` build script. Or if you insist on using CMake, you need to create a custom copy command that will copy your compiled module to the necessary package folder. — Roman Miroshnychenko, Aug 18 '17 at 09:47

score 6 · Accepted Answer · answered Aug 19 '17 at 07:51

As I said in my comment, binary Python modules are normally built with distutils/setuptools. For that you need to write setup.py script with all necessary options. Below is a very minimal example showing only basic things:

from setuptools import setup, Extension

setup(
    name = 'awesome',
    version = '0.0.1',
    packages = ['awesome_package']                     
    ext_modules = [Extension(
       'awesome_package.module2',
       ['src/module2.cpp']
    )]
)

Naturally, in setup.py you need to specify all your build options like header files, compiler flags etc.

If you insist on using CMake, you need to add a custom copy command to copy your compiled module inside your package. Something like this:

add_custom_command(TARGET module2 POST_BUILD
       COMMAND ${CMAKE_COMMAND} -E copy $<TARGET_FILE:module2>
       "${CMAKE_SOURCE_DIR}/awesome_package"
    )

Making a C++ module part of a Python package

1 Answers1