2

I have a Cython extension module for a Python project I want to install to the same namespace as the project on installation. When I try to specify in the extension to install it inside the package itself, it can't be found and imported. If I specify the extension goes in the root of the Python namespace, it works fine, but it's not in the module namespace I want. How do I get the Extension Module to be importable from the same namespace as the package itself?

I've made a simple test case for this question.

Folder Structure:

mypkg
├── foo
│   ├── __init__.py
│   └── barCy.pyx
└── setup.py

The .barCy.pyx file:

cpdef long bar():
    return 0

The setup.py code:

import distutils.extension
from Cython.Build import cythonize

from setuptools import setup, find_packages

extensions = [distutils.extension.Extension("foo.bar",
                                            ['foo/barCy.pyx'])]
setup(
        name='foo',
        packages=find_packages(),
        ext_modules=cythonize(extensions),
        )

The __init.py__ is empty.

I want to be able to do:

>>> import foo.bar as bar
>>> bar.bar()
0

Instead, I get

>>> import foo.bar as bar
ModuleNotFoundError: No module named 'foo.bar'

If I were to change the Extension("foo.bar",... to Extension("bar",..., then I can do import bar like it were a top level package. Although that is the expected behavior, it's not what I want as I only want this extension module to be accessible through the foo namespace.

Levi Naden
  • 41
  • 5
  • Does this answer your question? [Python: importing a sub‑package or sub‑module](https://stackoverflow.com/questions/12229580/python-importing-a-sub-package-or-sub-module) – funie200 Sep 25 '20 at 06:33
  • No, that question has to do with how to use ```import``` statements on already imported modules instead of having to resolve the full module path like you normally would. The problem here is the nested extension module is not importable at all and not part of the ```foo``` package namespace. The compiled ```.so``` file appears in the correct location, but it cannot be imported. – Levi Naden Sep 25 '20 at 13:06

1 Answers1

2

My Python interpreter was being run from the same folder as the setup.py script, so doing import foo was importing the local package, rather than the one installed in Python interpreter's site-packages directory. Because they have the same folder structure, the local directory was chosen as the superseding imported package.

Do testing/running from a different folder than the source.

Levi Naden
  • 41
  • 5
  • Wow -- this one bit me quite hard. Additional observation: when I ran `pip install -e .` my tests (in root dir) WOULD work, but after `pip install .` (parallel environment) my tests would throw Module Not Found, which confused the problem for me. This was the solution, which was confusingly not an issue for development (-e) install. – bunkerdive Jul 26 '23 at 20:05
  • A further observation -- running `pip install -e .` would show a new `.pyd` file getting created, but when running `pip install .` instead, I did NOT see that file so was thinking that `pip install` was somehow failing to run the cython compilation specified in my `setup.py` file, which even further confused matters for me. Again, everything was fine, I was just running tests from the wrong dir, as stated here. – bunkerdive Jul 26 '23 at 20:08