How to include .exe, apt-get and brew install files on a PyPi setup.py file

Question

I'm writing a setup.py file to install an OpenSource project using PyPi package. The thing is, this project requires to install as well outside apps (ghostscript, imagemagick, and tesseract). These apps have different ways to install depending on the platform (win, linux, or mac).

I wrote a file that when I execute python setup.py install install all of it. The problem is when I load the tar and whl files on PyPi and I execute pip install Gap-ML it is just installing the modules but it is not installing these apps.

Here the code on setup.py:

""" setup Gap-ML
Copyright, 2018(c), Andrew Ferlitsch
Autor: David Molina @virtualdvid
"""

from setuptools import setup, find_packages
from distutils.command.install import install
import os, sys, platform
import requests

##Install custom apps Ghostscript, Imagemagick, and Tesseract
def install_apps(app_name, app_path, url):
    """
      Install custom apps
      :param app_name: name of the app to install
      :param app_path: path on windows where the app should be installed
      :param url: url where the .exe file is located
    """
    #extract the app_name.exe from the url
    app = url.split('/')[-1]

    #verify if the software is already installed on windows
    if os.path.exists(app_path):
        print('{} already installed'.format(app_name))
    else:
        print('Download has started')

        #warning message to specify the correct path where to install the app
        if platform.architecture()[0] == '64bit':
            print('Please verify C:\\Program Files\\ is part of the path to install the app')
        else:
            print('Please verify C:\\Program Files (x86)\\ is part of the path to install the app')

        #download, install, and delete the app_name.exe
        r = requests.get(url, allow_redirects=True)
        open(app, 'wb').write(r.content)
        os.system(os.path.join(os.getcwd(),app))
        os.remove(app)

        #verify pythonpath to execute the apps from terminal
        if app_path not in sys.path:
            sys.path.append(app_path)
        print('{} has been installed successful'.format(app_name))

def main():
    #verify Operative System
    if sys.platform.startswith('win'):
        windows={'64bit':{1:{'app_name':'Ghostscript',
                             'app_path':'C:\\Program Files\\gs\\gs9.23\\bin\\',
                             'url':'https://github.com/ArtifexSoftware/ghostpdl-downloads/releases/download/gs923/gs923w64.exe'},
                          2:{'app_name':'Imagemagick',
                             'app_path':'C:\\Program Files\\ImageMagick-7.0.8-Q8',
                             'url':'https://www.imagemagick.org/download/binaries/ImageMagick-7.0.8-9-Q8-x64-static.exe'},
                          3:{'app_name':'Tesseract',
                             'app_path':'C:\\Program Files\\Tesseract-OCR',
                             'url':'https://digi.bib.uni-mannheim.de/tesseract/tesseract-ocr-w64-setup-v4.0.0-beta.1.20180608.exe'}
                         },
                 '32bit':{1:{'app_name':'Ghostscript',
                             'app_path':'C:\\Program Files (x86)\\gs\\gs9.23\\bin\\',
                             'url':'https://github.com/ArtifexSoftware/ghostpdl-downloads/releases/download/gs923/gs923w32.exe'},
                          2:{'app_name':'Imagemagick',
                             'app_path':'C:\\Program Files (x86)\\ImageMagick-7.0.8-Q8',
                             'url':'https://www.imagemagick.org/download/binaries/ImageMagick-7.0.8-9-Q8-x86-static.exe'},
                          3:{'app_name':'Tesseract',
                             'app_path':'C:\\Program Files (x86)\\Tesseract-OCR',
                             'url':'https://digi.bib.uni-mannheim.de/tesseract/tesseract-ocr-w32-setup-v4.0.0-beta.1.20180608.exe'}
                         },
                 'common':{1:{'app_name':'Ghostscript',
                              'url':'https://www.ghostscript.com/download/gsdnld.html'},
                           2:{'app_name':'Imagemagick',
                              'url':'https://www.imagemagick.org/script/download.php'},
                           3:{'app_name':'Tesseract',
                              'url':'https://github.com/UB-Mannheim/tesseract/wiki'}
                          }
                }

        OS=platform.architecture()[0]
        for i in range(1,4):
            try:
                app_name = windows[OS][i]['app_name']
                app_path = windows[OS][i]['app_path']
                url = windows[OS][i]['url']
                install_apps(app_name, app_path, url)
            except:
                app_name = windows['common'][i]['app_name']
                url = windows['common'][i]['url']
                print('{} Download files on {}'.format(app_name, url))

    elif sys.platform.startswith('linux'):
        #install Ghostscript:
        os.system('sudo apt-get update && sudo apt-get install ghostscript')
        #install ImageMagick:
        os.system('sudo apt-get install imagemagick')
        #install Tesseract:
        os.system('sudo apt-get install tesseract-ocr && sudo apt-get install tesseract-ocr-eng')

    elif sys.platform.startswith('darwin'):
        os.system('/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"')
        os.system('brew update')
        os.system('brew install ghostscript imagemagick tesseract')

##Setup config
#additional class to execute main() for custom install apps
class CustomInstall(install):
    def run(self):
        install.run(self)
        main()

#setup components
with open('README.md') as f:
    long_description = f.read()

install_requires=[
    'bs4',
    'numpy',
    'h5py',
    'imutils',
    'unidecode',
    'nltk',
    'requests',
    'opencv-python',
    'pillow',
    'pyaspeller']

tests_require=[
    'pytest',
    'pytest-cov']

package_data={'gapml': ['org-os/*', 'plan/*', 'tools/*', 'train/*']}

project_urls={"Documentation": "https://andrewferlitsch.github.io/Gap/",
              "Source Code": "https://github.com/andrewferlitsch/Gap"}

#https://pypi.org/pypi?%3Aaction=list_classifiers
classifiers=[
    'Development Status :: 3 - Alpha',
    'Intended Audience :: Healthcare Industry',
    'Topic :: Text Processing',
    'License :: OSI Approved :: Apache Software License',
    'Operating System :: Microsoft :: Windows',
    'Operating System :: MacOS',
    'Operating System :: POSIX :: Linux',
    'Programming Language :: Python :: 3',
    'Programming Language :: Python :: 3.0',
    'Programming Language :: Python :: 3.1',
    'Programming Language :: Python :: 3.2',
    'Programming Language :: Python :: 3.3',
    'Programming Language :: Python :: 3.4',
    'Programming Language :: Python :: 3.5',
    'Programming Language :: Python :: 3.6',
    'Programming Language :: Python :: 3.7']

setup(
    name='Gap-ML',
    version='0.9.2',
    description='NLP and CV Data Engineering Framework',
    author='Andrew Ferlitsch',
    author_email='aferlitsch@gmail.com',
    license='Apache 2.0',
    url='https://github.com/andrewferlitsch/Gap',
    project_urls=project_urls,
    long_description=long_description,
    packages=find_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"]),
    install_requires=install_requires,
    tests_require=tests_require,
    package_data=package_data,
    cmdclass={'install': CustomInstall},
    classifiers=classifiers
)

How can I install these apps from PyPi? What am I missing? :/

We are a young OpenSource Project for Machine Learning and Natural Language Processing. If someone want to contribute you are welcome to join us. :)

Repo on GitHub
Documentation

Any help will be apreciatte, Thanks!

There are several issues with this approach: `apt` is not a default package manager for Linux distros - what if your package is installed on CentOS (`yum`), Fedora (`dnf`), SUSE (`zypper`) etc? On MacOS, you don't check whether `brew` is already installed, also whether it's configured correctly, adding library paths to `DYLD_LIBRARY_PATH` if necessary. This can quickly end in a complex and hard-to-maintain solution, not sure you really want that. — hoefling, Aug 18 '18 at 10:02
In general, `setup.py` should only declare Python dependencies. If you need non-Python deps, check in your setup script whether they are available. If not, abort the installation, printing an info message describing how to install required stuff on the current platform. Another alternative is to bundle external shared libs into the wheel itself: check out [this question](https://stackoverflow.com/questions/47042483/how-to-build-and-distribute-a-python-cython-package-that-depends-on-third-party) for an example of how to produce such wheels for Linux/MacOS with `auditwheel`/`delocate`, resp. — hoefling, Aug 18 '18 at 10:11
Hey guys thank you so much. I'm pretty new with all those things as you can see. I'll talk with my team and fix our `setup.py` and documentation! — virtualdvid, Aug 18 '18 at 15:14
I don't have enough knowledge to write up a separate answer, but maybe look into Conda. They run a separate package manager which _does_ know how to install system libraries and arbitrary binaries etc as part of its installation process. — tripleee, May 12 '23 at 04:29
Other than that, you need a separate mechanism for each platform you want to support (create `.deb` packages for each Python dependency which isn't already separately packaged with a sufficiently up-to-date version for Debian-based platforms; similarly for `.rpm` distros for various architectures; similarly for Homebrew or some other package manager for MacOS; sacrifice to Satan for Windows). — tripleee, May 12 '23 at 04:29

score 2 · Accepted Answer · answered Aug 17 '18 at 21:28

Because you uploaded a wheel (.whl) file to PyPI, pip install ... will install from that wheel, and wheels do not in any way support what you're trying to do. In theory, you could get this to "work" by only uploading a .tar.gz or .zip file to PyPI so that pip installs from that, but this would require the user to always install your package with administrator privileges, which is a bad idea that goes against recommended practice and doesn't always work (e.g., users would be unable to install your package in a virtualenv). The proper thing to do is to not install any system packages in your setup.py; instead, document the needed packages and how to install them in your project's README.

I noticed that before it only works with the `tar` file but it depends on the OS. We had it like you say all the instructions in the README.md file. I had the idea to avoid this and make an easy install for our future users. It's fine I'll share all this useful information with my team and get the best approach. Thanks! — virtualdvid, Aug 18 '18 at 15:19

How to include .exe, apt-get and brew install files on a PyPi setup.py file

1 Answers1

Linked

Related