0

I need to make a Python package openface, that will act as an API of some existing C++ research code OpenFace, that I don't own.

I have forked the existing code base into openface-backend, and rewrote the installation script install.sh to not require sudo, such that after cloning the repository and installing the system requirements, it compiles the code and download assets build/bin. At that point, one can run ./bin/foo --bar /path/to/abc --baz /path/to/xyz and it all works.

Regarding the Python package, for a starter it will be very basic. For the executable ./bin/foo, I will have a corresponding function foo():

import subprocess
import shlex

BINARY_PATH = "/path/to/bin/foo"

def foo(bar: str, baz: str):
    args = f"{BINARY_PATH} --bar {bar} --baz {baz}"
    try:
        process = subprocess.run(
            args=shlex.split(args),
            check=True,
            stdout=subprocess.PIPE,
            stderr=subprocess.STDOUT,
            text=True,
        )
        print(process.stdout)
    except subprocess.CalledProcessError as error:
        print(error.stdout)

I would like that when installing the package with pip, such as pip install 'git+https://github.com/GuillaumeRochette/openface.git', the openface-backend repository would be downloaded and compiled, and that the binaries are to be moved somewhere in the package.

To try and make it work, I have followed the instructions from another answer.

This is my setup.py:

from setuptools import setup
from setuptools.command.develop import develop
from setuptools.command.install import install

import subprocess
import shlex


def compile():
    args = (
        'bash -c "'
        "git clone https://github.com/GuillaumeRochette/openface-backend.git"
        " && "
        "cd openface-backend"
        " && "
        "bash install.sh"
        " && "
        "mv build/bin .."
        " && "
        "cd .."
        " && "
        "rm -rf openface-backend"
        '"'
    )
    try:
        process = subprocess.run(
            args=shlex.split(args),
            check=True,
            stdout=subprocess.PIPE,
            stderr=subprocess.STDOUT,
            text=True,
        )
        print(process.stdout)
    except subprocess.CalledProcessError as e:
        print(e.stdout)


class CustomDevelop(develop):
    def run(self):
        develop.run(self)
        compile()


class CustomInstall(install):
    def run(self):
        install.run(self)
        compile()


if __name__ == "__main__":
    setup(
        cmdclass={
            "develop": CustomDevelop,
            "install": CustomInstall,
        },
    )

While the code is getting downloaded and compiled when doing python setup.py install within that repository, this is not happening when doing pip install 'git+https://github.com/GuillaumeRochette/openface.git'. I don't know if it is the pyproject.toml which is missing something, or if my setup.py needs some changes or if this is something totally different.

sinoroc
  • 18,409
  • 2
  • 39
  • 70
AcideBase
  • 11
  • 1
  • As far as I know, by default *pip* does not run the `python setup.py install` command (nor `python setup.py develop`). -- Generally the advice you followed seems to be very outdated. I would not use this technique at all. I would need to take some more time to try to understand what you are trying to achieve to be able to give better, more up-to-date advice. -- This is quite a big undertaking anyway, you will need to do a lot of research invest quite a lot of time. -- I encourage you to ask here to get some advice on how to get started properly: https://discuss.python.org/c/packaging/14 – sinoroc Apr 01 '23 at 16:33
  • Maybe you should look for other cases of Python bindings for a C++ library, if possible one that is state of the art, and try to get inspired from them. -- If you are into this kind of communication system, there is a Discord server for _PyPA_ which is the org dedicated to Python packaging: https://discuss.python.org/t/theres-a-pypa-discord-now/8515 and you might be able to get help there to get you started. – sinoroc Apr 01 '23 at 16:42

0 Answers0