0

To get close to a MWE, I have an example project that illustrates my problem. The project tries to perform a simple operation on a wrapped C++ class, in this case a PCLHeader object.

I am pretty sure the problem lies in the fact that my cmake call includes two .pyx files. cython_add_module(test_cython common.pyx test_cython.pyx).

Based on the documentation in UseCython.cmake, which is part of the project, this call should be okay as far as I can tell.

When I build this project, then try to import the resulting library, Python fails to import the first time only. 

In [1]: import test_cython
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-158d7481838a> in <module>()
----> 1 import test_cython

common.pxd in init test_cython (/Users/phil/devel/test_cython/build/test_cython.cxx:1424)()

ImportError: No module named common

In [2]: import test_cython

In [3]:

As an aside, if anybody could steer me toward a better design if this is wrong in some way, I would appreciate it. For example, I cannot understand why this line causes compile errors based on my setup.

phil0stine
  • 303
  • 1
  • 13

1 Answers1

2
cython_add_module(test_cython common.pyx test_cython.pyx)

This compiles common and test_cython into a single module (test_cython.so or something similar). When you import that however it will run the initialization for test_cython but not for common. (When Python loads a compiled module it runs a function called PyInit_<modulename> or init_<modulename> depending on on if you're running Python 2 or 3. Although PyInit_common exists, it isn't called since you're importing test_cython)

At some point within test_cython you then attempt to import from common. The Python import mechanism is based around each module being a separate file - it first searches through a list of different file types(common.py, common.pyc, common.so etc.) and finds none of them. Therefore an ImportError is raised.

Cython doesn't support compiling multiple modules into a single .so file. (You can occasionally get away with things that look similar though, provided they only use C interfaces - it still isn't a great idea even if it seems to work). What you need to do is compile test_cython.pyx to one module and common.pyx to a second module:

cython_add_module(test_cython test_cython.pyx)
cython_add_module(common common.pyx)
DavidW
  • 29,336
  • 6
  • 55
  • 86
  • With respect to your compile time error - it's probably failing to deduce types correctly and you can probably fix that by specifying them. However I don't know what the error actually is. – DavidW Sep 23 '17 at 08:05
  • I tried your solution during debug but I think I followed it up with a call to `target_link_libraries` which failed. It didn't occur to me that it would just work without linking, but it makes sense in the context of importing. Thanks, I will test shortly – phil0stine Sep 23 '17 at 12:17
  • Worked, as expected. I think you need to edit for me to upvote (I think my child errantly downvoted), but I will accept in either case. As for the compile error, I am getting the classic `Cannot convert Python object to 'PCLHeader'`, even for the following line `cdef PCLHeader hw = HeaderWrap(h).header_cpp[0]`. In any case thank you. – phil0stine Sep 24 '17 at 05:15
  • I think you need to `cimport HeaderWrap` from `common` so that Cython is aware that it is a `cdef` type and thus can use its `cdef` attributes. I'm not in a position to test that right now though – DavidW Sep 24 '17 at 06:43