I'm currently trying to pack a module that uses precompiled *.pyd files from a swig routine.
The process for the user is supposed to be:
- install base library (C, C++); directories linked in the environment variables; here are also the *.pyd files.
- get python package; open dir from python environment (be it conda or else) and run "pip install ."
- enjoy module....
What I did:
- I generated the manifest file to include the *.pyd files from setup.py based on the environmental path from the installation.
- I checked and upon installation of the module, the files are listed in the sources text file of the egg info.
But there are no *.pyd files in the module.
This is the way I assumed was correct, but I also tried many other ways (data option in setup specification, etc.), but nothing worked so far.
What did I do wrong? On research, it seems like C related files are deleted after installation, but I thought the manifest defined files are save.
Edit: setup.py added.
from pathlib import Path
from setuptools import setup, find_packages, Extension
import os
import shutil
name = 'mypackage'
#define requirements
REQUIREMENTS = {
# Installation script (this file) dependencies
'setup': [
'setuptools_scm',
],
# Installation dependencies
# Use with pip install . to install from source
'install': [
'Cpython',
'setuptools < 64',
'numpy >= 1.23',
'matplotlib',
'DateTime',
'psutil',
'xarray',
'PyYAML',
'scipy',
'PySimpleGui'
],
}
#check for installed C library
lib_dir = ""
if 'BASEPACKAGE_IN_C' in os.environ:
lib_dir = os.getenv('BASEPACKAGE_IN_C')
print('BASEPACKAGE_IN_C found at {}!'.format(lib_dir))
else:
Exception('BASEPACKAGE_IN_C does not seem to exist on this machine! Make sure that the environment variable BASEPACKAGE_IN_C is set.')
# define function to make manifest file
def __createManifest__(subdirs):
"""inventory all files in path and create a manifest file"""
current = os.path.dirname(__file__)
relative_paths = [os.path.relpath(path, current) for path in subdirs]
with open(os.path.join(current, "MANIFEST.in"), "w") as manifest:
manifest.writelines("recursive-include {} *".format(" ".join(relative_paths)))
# check for interface layer directory
add_il = Path(lib_dir).parents[0].joinpath("sdk", "my_package_pyd_dir")
il_dest = os.path.join(os.path.dirname(__file__), "pyil" + os.sep)
if not os.path.exists(il_dest):
os.makedirs(il_dest)
if os.path.exists(add_il):
print('Python SDK interface layer found at {}!'.format(add_il))
for root, dirs, files in os.walk(add_il):
for file in files:
#copy files locally
shutil.copy(os.path.join(root, file), il_dest)
else:
Exception('Python SDK interface layer does not seem to exist on this machine! Make sure that the BASEPACKAGE_IN_C SDK is '
'properly installed.')
# make manifest file
__createManifest__([il_dest])
#standard setup call
setup(
name=name,
python_requires='>= 3.9',
version='0.1',
packages=find_packages(),
url='',
license='',
author='Ben',
author_email='',
description='BASEPACKAGE_IN_C Python SDK. Linked to the BASEPACKAGE_IN_C installation at {}.'.format(lib_dir),
setup_requires=REQUIREMENTS['setup'],
install_requires=REQUIREMENTS['install'],
include_package_data=True,
)
The setup.py generates the manifest.in with the following line:recursive-include pyil *
I also tried include pyil * or specifying the ending recursive-include pyil *.pyd or combinations thereof.
The Sources.txt file looks like that:
MANIFEST.in
setup.py
mypackage/moduleClass1.py
mypackage/moduleClass2.py
mypackage/moduleClass3.py
mypackage/__init__.py
mypackage.egg-info/PKG-INFO
mypackage.egg-info/SOURCES.txt
mypackage.egg-info/dependency_links.txt
mypackage.egg-info/requires.txt
mypackage.egg-info/top_level.txt
pyil/_pyil.pyd
So it is working so far. I tried with different filetypes in pyil, and they all worked beside the pyd file.
