8

I have a Python package in python3.x I've been writing up which interacts with two C functions in a C library. At the moment, I've been using ctypes whereby I link directly to the shared library *.so, and then use the python script to interact with this.

https://docs.python.org/3/library/ctypes.html#module-ctypes

I'm now confused how to distribute this python package whereby users would install the python package on github or via pip. Somehow, upon installation, the C library would need to be downloaded, unpacked, and compiled.

The issue is, the C library contains several dependencies to other C libraries; I've currently statically linked these libraries, and the entire C library now installs via cmake.

(1) Is it possible to install the C library as it installs now, via cmake? It appears this should be done entirely in setup.py, correct?

(2) I'm confused by the distutils/setuptools issue in this case. Based on this link, https://docs.python.org/3/extending/building.html#building

the correct manner to do this is via distutils, not setuptools. However, setuptools is generally the recommended ways to create/distribute python packages in 2018. Is the documentation above outdated?

ShanZhengYang
  • 16,511
  • 49
  • 132
  • 234
  • 1
    Anything you can do with `distutils` can be done with `setuptools` - `from setuptools import setup, Extension` – SethMMorton Jun 07 '18 at 03:20
  • 1
    Here is an example of invoking CMake from the setup script: https://stackoverflow.com/questions/42585210/extending-setuptools-extension-to-use-cmake-in-setup-py/48015772#48015772 – hoefling Jun 07 '18 at 09:22
  • Building the library code at module install time, would require the user to have a *C* compiler (typycally *gcc* for *Ux*, *VStudio* for *Win*). Using *cmake* would make it an **additional** dependency required at install time. You could also distribute some prebuilt binary versions (at least for most common setups), like e.g https://pypi.org/project/psycopg2/#files. Additional work on your side but less on everyone who wants to use the module (waaaay less work overall, and more users likely to install and use your module) – CristiFati Jun 08 '18 at 08:00
  • @CristiFati I'm confused about how stable these binary versions would be though. Would they be stable on all flavors/versions of Linux, MacOS, Windows, etc.? – ShanZhengYang Jun 08 '18 at 14:50
  • What do you mean by stable? I gave an example that (I think) speaks for itself (there are many more). But it's kind of hard to work with stuff like "a library that depends of other libraries". – CristiFati Jun 08 '18 at 17:13
  • 1
    This does not answer the OP's question but using cffi (instead of ctypes) to convert a static C lib into a pyo object is a better solution. – Major Eccles Nov 25 '18 at 12:25
  • @MajorEccles Thanks for the input---"better solution": could you elaborate on this a bit more? – ShanZhengYang Nov 25 '18 at 18:53
  • 1
    @ShanZhengYang Using ctypes involves some mental mapping between the internal C pointers and addresses to Python which in my experience is very easy to get wrong. Additionally one has to also distribute the .so file with the python wheel and all that entails. I got arund this by using cffi - first step is to create a static lib (*.a) with an associated header file (*.h). Then one builds a pyo file directly from this using cffi. In python code one imports cffi and can get to the C functions by using cffi.get_something(). https://cffi.readthedocs.io/en/latest/overview.html#main-mode-of-usage – Major Eccles Dec 02 '18 at 13:11
  • @MajorEccles This sounds far easier than types! Thanks! I'll be sure to ask SO questions if I get stuck :) – ShanZhengYang Dec 04 '18 at 00:16

0 Answers0