10

I'm developing a Python application and in the process of branching off a release. I've got a PyPI server set up on a company server and I've copied a source distribution of my package onto it.

I checked that the package was being hosted on the server and then tried installing it on my local development machine. I ended up with this output:

$ pip3 install --trusted-host 172.16.1.92 -i http://172.16.1.92:5001/simple/ <my-package>
Collecting <my-package>
  Downloading http://172.16.1.92:5001/packages/<my-package>-0.2.0.zip
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\<me>\AppData\Local\Temp\pip-build-ubb3jkpr\<my-package>\setup.py", line 9, in <module>
        import appdirs
    ModuleNotFoundError: No module named 'appdirs'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\<me>\AppData\Local\Temp\pip-build-ubb3jkpr\<my-package>\

The reason is that I'm trying to import a third-party library appdirs in my setup.py, which is necessary for me to compute the data_files argument to setup():

try:
    from setuptools import setup
except ImportError:
    from distutils.core import setup

import os
from collections import defaultdict

import appdirs
from <my-package>.version import __version__ as <my-package>_version

APP_NAME = '<my-app>'
APP_AUTHOR = '<company>'
SYSTEM_COMPONENT_PLUGIN_DIR = os.path.join(appdirs.user_data_dir(APP_NAME, APP_AUTHOR), 'components')

# ...

setup(
    # ...
    data_files=component_files,
)

However, I don't have appdirs installed on my local dev machine and I don't expect the end users to have it either.

Is it acceptable to rely on third-party libraries like this in setup.py, and if so what is the recommended approach to using them? Is there a way I can ensure appdirs gets installed before it's imported in setup.py, or should I just document that appdirs is a required package to install my package?

Tagc
  • 8,736
  • 7
  • 61
  • 114
  • Have you tried using `setup_requires`? See https://pip.readthedocs.io/en/1.4.1/cookbook.html#controlling-setup-requires – Peter Brittain Jun 04 '17 at 16:37
  • @PeterBrittain Are there any examples of how to use it? Should I do something like make two `setup()` calls, one near the top of `setup.py` with just `setup_requires=['appdirs']` and the one I have already at the bottom? – Tagc Jun 05 '17 at 07:11
  • It's not common... You can find a simple example in https://stackoverflow.com/questions/37471313/setup-requires-with-cython. That begins to show the sorts of hoops mentioned in https://www.python.org/dev/peps/pep-0518/ which highlights another option for doing this with pip that should be available soon. – Peter Brittain Jun 05 '17 at 23:32

3 Answers3

3

I'm ignoring licensing issues in this answer. You definetly need to take these into account before you really do a release.

Is it acceptable to rely on third-party libraries like this in setup.py

Yes, it is acceptable but generally these should be minimized, especially if these are modules which have no obvious use for the end-user. Noone likes to have packages they don't need or use.

what is the recommended approach to using them?

There are basically 3 options:

  • Bootstrap them (for example use pip to programmatically install packages). For example setuptools provides an ez_setup.py file that can be used to bootstrap setuptools. Maybe that can be customized to download and install appdirs.

  • Include them (especially if it's a small package) in your project. For example appdirs is basically just a single file module. Pretty easy to copy and maintain in your project. Be very careful with licensing issues when you do that!

  • Fail gracefully when it's not possible to import them and let the user install them. For example:

    try:
    import appdirs
except ImportError:
    raise ImportError('this package requires "appdirs" to be installed. '
                      'Install it first: "pip install appdirs".')
MSeifert
  • 145,886
  • 38
  • 333
  • 352
  • Looks like `appdirs` is licensed under MIT, so of the three the second option seems most appealing. Ideally `setup_requires` (which @PeterBrittain recommends) looks like the best solution, but I'm not exactly sure how to use it. – Tagc Jun 05 '17 at 07:12
  • Well, it depends on the license, the size, the stability and if you also need it as runtime-dependency to decide which approach is the "most appealing". I personally don't like `setup_requires` because it's annoying (impossible) to use with some ways of locally installing a package (I think `python setup.py install`, `python setup.py develop` or `pip install .` won't work). – MSeifert Jun 06 '17 at 00:20
  • `appdirs` seems to resolve paths weirdly when run as a local module, so I went with the third option. – Tagc Jun 06 '17 at 09:41
3

You could use pip to install the package programmatically if the import fails:

try:
    import appdirs
except ImportError:
    import pip
    pip.main(['install', 'appdirs'])
    import appdirs

In some circumstances you may need to use importlib or __import__ to import the package after pip.main or referesh the PATH variable. It could also be worthwhile to include a verification if the user really wants to install that package before installing it.

I used a lot of the examples from "Installing python module within code" and I haven't personally tried used this in setup.py files but it looks like it could be a solution for your question.

MSeifert
  • 145,886
  • 38
  • 333
  • 352
  • I'm curious why this was downvoted. This seems like a painless and functional solution to the problem that I personally was having. – Chris Allen Lane Feb 07 '19 at 16:39
1

You can mention install_requires with the dependencies list. Please check the python packaging guide here. Also you can provide a requirements.txt file so that it can be run at once using "pip install -r"

Venu
  • 63
  • 1
  • 11
  • I'm not sure how `install_requires` would help, did you mean [`setup_requires`](http://setuptools.readthedocs.io/en/latest/setuptools.html#new-and-changed-setup-keywords)? However in my experience it's tricky to get right. Especially if one can't garantuee that `setuptools` is installed and given the `try: from setuptools import setup` that's not garantueed. – MSeifert Jun 04 '17 at 17:40
  • If am not wrong, setup_requires just downloads the dependencies but install_requires installs them, I agree setup tools should be imported. – Venu Jun 05 '17 at 04:48
  • @Venu: it is fine to require `setuptools` to be able to install your package. If you use `pip` to install, `setuptools` is bundled already. I'd just use `setuptools` and `setup_requires` here. – Martijn Pieters Jun 05 '17 at 10:34