10

I have a python module module.pyd that works pretty fine once it is put manually onto the site-packages of python installation folder.

The problem starts when I upload my solution to a cloud enviroment, the buildpack requests that I pass every module as a package to be installed with pip install module. I ve created a folder with a simple __init__.py file that just imports everything of the module.pyd so that my module is treated like a folder.

Then I read here http://peterdowns.com/posts/first-time-with-pypi.html how to upload my own module and I succeeded, but when I install my module, the module.pyd file is not copied. I also tried to install it direct by the repository pip install git+repository but the same thing happened.

I have read here https://docs.python.org/2/distutils/sourcedist.html#specifying-the-files-to-distribute that I might have to explicitly say I want to copy *.pyd files in a MANIFEST.in file, I have done it, but it seems not working yet.

I currently using python 2.7.10

I am new on python so I d appreciate you guys help

Lucas Vieira de Oliveira
  • 101
  • 1
  • 1
  • 3
  • Not much of this makes sense, unfortunately. Why do you not want to distribute the actual .py files? Why do you need to put it in site-packages, rather than in your project itself, if it's your own code? – Daniel Roseman May 04 '16 at 15:03
  • @daniel-roseman Actually I dont have the py files of this module (it is a C++ compilation I think). I just want to place it as a distribution or at least in a repository so the buildpack of the cloud enviroment can access it. But once it install the module, the '__init__.py' file is dowloaded, but not the pyd file. I already tried to upload it together with the solution, but I dont know why, it doesn't work – Lucas Vieira de Oliveira May 04 '16 at 15:15

1 Answers1

10

Just use the MANIFEST.in:

recursive-include module *.pyd

This will include all pyd files in the module directory.

Your package layout should be the following:

module/
--- __init__.py
--- _module.pyd
--- module.py
MANIFEST.in
README.rst
setup.py

And don't forget to add include_package_data=True in setup() in your setup.py in order to force using MANIFEST.in when building wheels and win32 installers (else MANIFEST.in will only be used for source tarball/zip).

Minimal example of setup():

README_rst = ''
with open('README.rst', mode='r', encoding='utf-8') as fd:
    README_rst = fd.read()

setup(
    name='module',
    version='0.0.1',
    description='Cool short description',
    author='Author',
    author_email='author@mail.com',
    url='repo.com',
    packages=['module'],
    long_description=README_rst,
    include_package_data=True,
    classifiers=[
        # Trove classifiers
        # The full list is here: https://pypi.python.org/pypi?%3Aaction=list_classifiers
        'Development Status :: 3 - Alpha',
    ]
)
gaborous
  • 15,832
  • 10
  • 83
  • 102
  • Can you elaborate on why you're using `module.py` and `_module.pyd` (why add the .py file and rename the .pyd file with an underscore)? – florisla Aug 09 '17 at 13:56
  • @florisla ah sorry it might indeed be confusing, `_module.pyd` is here a SWIG-generated Python module interface to a C++ program, so there is no relation between `_module.pyd` and `module.py`, it would probably be less confusing to rename the pyd file to `_core.pyd`. – gaborous Aug 09 '17 at 18:52
  • Then why keep the underscore? You want to make the *.pyd file public, right? – florisla Aug 11 '17 at 19:09
  • We wanted to provide a more pythonic interface with docstrings and default named keywords, so the py file basically just redefined functions and classes of the pyd :-) Also the py file managed cross platform compatibility by loading the appropriate pyd, so in my concrete case i had multiple pyd files generated for various platforms. – gaborous Aug 12 '17 at 08:20
  • @gaborous ... and you shipped every `.pyd` file to every platform? Why not just `wheel` the right `.pyd` to the right platform? –  Aug 27 '19 at 16:30
  • @TylerGubala For simplicity in the package definition, but yes indeed once it works like this, it's possible to make specific definitions for each platform. I'm not yet much used to Python 3, so it might be easier to do so now. – gaborous Sep 02 '19 at 20:10
  • This was the only solution I could find that actually worked – Geordie Oct 06 '20 at 23:49