Distributing a python package with cython files (.pdx) within

Question

Question: Distributing a python package with cython files (.pdx) within.

I am trying to package a python module with both .py and .pdx files within.

Here is the folder structure:

sosecrets: Root
|___ src
    |___ sosecrets
        |___ __init__.py
        |___ build.py
        |___ secretdicts.py
        |___ secrets.pyi
        |___ secrets.pyx
|___ setup.md
|___ pyproject.toml

I am using poetry to attempt to build this package.

The core extension type is written in secrets.pdx and other types based on it are written in secretdicts.py.

I ran poetry build at the sosecrets Root level, this is what I get:

poetry build

Preparing build environment with build-system requirements poetry-core, cythonBuilding sosecrets (0.1.4)
A setup.py file already exists. Using it.
Traceback (most recent call last):
  File "F:\py_projects\sosecrets\setup.py", line 29, in <module>
    from src.sosecrets.build import *
  File "F:\py_projects\sosecrets\src\sosecrets\__init__.py", line 1, in <module>
    from .secretdicts import ImmutableSecretMapping, MutableSecretMapping
  File "F:\py_projects\sosecrets\src\sosecrets\secretdicts.py", line 1, in <module>
    from .secrets import Secret
ModuleNotFoundError: No module named 'src.sosecrets.secrets'

Command '['C:/Users/.../AppData/Local/Temp/tmp289id5kw/.venv/Scripts/python.exe', 'F:\\py_projects\\sosecrets\\setup.py', 'build', '-b', 'F:\\py_projects\\sosecrets\\build']' returned non-zero exit status 1.

In setup.py:

# -*- coding: utf-8 -*-

package_dir = \
{'': 'src'}

packages = \
['sosecrets']

package_data = \
{'': ['*'],
 'sosecrets': ['build/lib.win-amd64-cpython-310/src/sosecrets/*',
               'build/temp.win-amd64-cpython-310/Release/*']}

setup_kwargs = {
    ... omitted ...
}
from src.sosecrets.build import *
build(setup_kwargs)

In build.py:

import os
# https://stackoverflow.com/questions/63679315/how-to-use-cython-with-poetry
def get_ext_paths(root_dir, exclude_files):
    """get filepaths for compilation"""
    paths = []

    for root, dirs, files in os.walk(root_dir):
        for filename in files:
            if os.path.splitext(filename)[1] not in  ['.py', '.pyi', '.pyx']:
                continue

            file_path = os.path.join(root, filename)
            if file_path in exclude_files:
                continue

            paths.append(file_path)
    print("Paths: ", paths)
    return paths

import src.sosecrets

ROOT_DIR = os.path.dirname(src.sosecrets.__file__)
print("ROOT_DIR: ", ROOT_DIR)

# See if Cython is installed
try:
    from Cython.Build import cythonize
# Do nothing if Cython is not available
except ImportError:
    # Got to provide this function. Otherwise, poetry will fail
    def build(setup_kwargs: dict):
        pass
# Cython is installed. Compile
else:
    print("Cython is installed")
    from setuptools import Extension
    from setuptools.dist import Distribution
    from distutils.command.build_ext import build_ext

    # This function will be executed in setup.py:
    def build(setup_kwargs: dict):
        # The file you want to compile
        extensions = get_ext_paths(ROOT_DIR, list())
        print("Extensions are: ", extensions)

        # gcc arguments hack: enable optimizations
        os.environ['CFLAGS'] = '-O3'

        # Build
        setup_kwargs.update({
            'ext_modules': cythonize(
                extensions,
                language_level=3,
                compiler_directives={'linetrace': True},
            ),
            'cmdclass': {'build_ext': build_ext}
        })

The actual Github repo is located at here

How do I package a .py and .pyx module together?

Distributing a python package with cython files (.pdx) within

0 Answers0