45

I am creating a setup.py to distribute my application. This application has a number of dependencies which can be installed via pip, it also has some custom dependencies which can not be installed from PyPI.

So, I have created a custom_package_0.1.whl which will be included into the distribution and must be installed as a dependency after setup.py installs everything from install_requires.

Imagine the following app structure:

my_app/
    win_deps/custom_package_0.1.whl
    my_app/
        __init__.py
        main.py
        setup.py
        setup.cfg

How do I do that?

minerals
  • 6,090
  • 17
  • 62
  • 107
  • I had a similar problem and found a satisfying answer [here](http://stackoverflow.com/questions/26059111/build-a-wheel-egg-and-all-dependencies-for-a-python-project) – Julia Niewiejska Mar 23 '17 at 15:53
  • 7
    It's a shame for such widely known language to not support local dependencies in a non hacky way... – kaiser Jan 15 '21 at 11:47

5 Answers5

30

There is a new technique (Since version 19.1) called Direct references. Just pretend like your file is hosted on localhost.

from setuptools import setup

path_to_my_project = "/home/user/projects/my_package"  # Do any sort of fancy resolving of the path here if you need to


setup(# ... other arguments
      install_requires=[f"my_package @ file://localhost/{path_to_my_project}#egg=my_package"]
      )
RunOrVeith
  • 4,487
  • 4
  • 32
  • 50
  • This does not seem to work with Python 3.10 and setuptools 59. I get the error message `Couldn't find index page for ' (maybe misspelled?)` with either `localhost` or `localbuilds` specified. – Richard Neumann Jan 10 '22 at 10:20
  • Nothing in the [release history](https://setuptools.pypa.io/en/latest/history.html) suggests that something changed here – RunOrVeith Jan 11 '22 at 09:17
  • Yeah, I think that this is an issue in setuptools, rather than pip. I stumbed over this here: https://stackoverflow.com/questions/70650046/pip-use-project-folder-as-installation-source – Richard Neumann Jan 11 '22 at 11:59
9

it is possible but not sure what setuptools version you should use. steps:

in setup.py

setup(
  ...,
  install_requires=['my-package'],
  dependency_links=[
    # location to your egg file
    os.path.join(os.getcwd(), 'deps', 'my_package-1.0.0-py3.5.egg')
  ]
)

important thing is that your location should not pass URL pattern test and egg file name should have structure <package_name_with_no_hyphens>-<version>-<py_version>.egg

wiesiu_p
  • 558
  • 7
  • 6
  • It seems like this wouldn't work if you haven't already installed `my-package` because there is no .egg until you install it, right? – medley56 May 01 '20 at 16:36
  • @medley56 I don't think that you need to install `my-package` with conda or something different. setuptools reads it from local location I guess – wiesiu_p Jul 03 '20 at 08:21
  • 3
    [dependency_links has been removed](https://github.com/Applied-GeoSolutions/gips/issues/246) – betontalpfa Mar 25 '21 at 15:27
  • 2
    this does not work for newer pip versions, scroll to my answer for something that works with pip >= 19.1 – RunOrVeith May 18 '21 at 08:19
4

Based on @RunOrVeith answer above this works for a local wheel file using a relative path. So it can be used on various hosts to install a third party package. Works on Windows and Unix.

setup(# ... other arguments
      install_requires=[
          f"my-lib @ file://localhost/{os.getcwd()}/libs/my_lib-xxx-none-any.whl"
      ]
)
HeyMan
  • 1,529
  • 18
  • 32
  • I'm not sure that if depending on 'cwd' is the correct way, what if cwd changes ? – luochen1990 Aug 12 '22 at 05:14
  • Since the ".../libs" is relative to the setup.py seems ok to me. If you move the entire project to another folder run "pip install -e ." and you should be fine. – HeyMan Aug 12 '22 at 08:19
3

Extending wiesiu_p's answer, you can install the dependency by linking to its source directory, which has its own setup.py.

Assume you have the source files of your dependency my-dependency, and the root of my-dependency has its own setup.py. In your application's setup.py:

setup(
  ...,
  install_requires=['other-dependency','my-dependency'],
  dependency_links=[
    # location to your my-dependency project directory
    ''.join(['file:\\', os.path.join(os.getcwd(), 'path', 'to', 'my-dependency#egg=my-dependency-1.0')])
  ]
)

Now if you run python setup.py install with your application's setup.py, it will install my-dependency.

Anfernee
  • 1,445
  • 1
  • 15
  • 26
  • 1
    I know this comment is old but I cannot get it to work. I tried with both `files:\\` as you have it and `files://` which I think is correct? doing `pip install --find-links= my_package` works, but not when I put it in setup.py – jmerkow Jan 31 '20 at 21:28
  • I've added some clarification to the answer. I hope it helps. – Anfernee Feb 02 '20 at 17:15
  • 1
    Note that this might fail if the local dependency itself has local dependencies, because the getcwd() might be called from site-packages. – nrlakin May 27 '20 at 14:02
  • 3
    This does not work for me: it says that `ERROR: No matching distribution found for my-dependency` when I do `pip install -e .`. Even if I do `python setup.py install` I get an error `error: Could not find suitable distribution for Requirement.parse('my-dependency')`. I'm using pip 20.3.3. – nbro Jan 07 '21 at 19:47
  • See my answer for how to get this to work with newer pip versions – RunOrVeith Jan 26 '21 at 13:35
  • Should that be `file://` instead of `file:\\ `? – Sören May 05 '22 at 18:36
  • @Sören Pretty sure it's ```file:\\```. Did that not work for you? – Anfernee May 06 '22 at 02:05
  • I've never seen an URI written that way, that's all. – Sören May 06 '22 at 07:09
-4

There are several options that you can choose from:

  1. Upload your package to some server, and provide the URL with dependency_links.
  2. You could put your python package inside of your my_app package and link it with packages parameter, instead of using the wheel file.
  3. A more hacky way would be to use the setuptools api, and install the package by yourself.
Gal Ben David
  • 410
  • 4
  • 10
  • 1
    For `2`, how should I do that? I moved "win_deps" into "my_app" and added `packages=(['win_deps'])` but running `python2 setup.py install` does not automatically install all the `.whl` packages from "win_deps" directory – minerals Feb 27 '16 at 14:34
  • 1
    What I meant in 2 was that instead of supplying the `whl` package, you could move the entire package directory `custom_package_0.1` into this package directory. – Gal Ben David Feb 27 '16 at 21:12
  • oh, you mean just move `custom_package_0.1` source files into `win_deps`? – minerals Feb 27 '16 at 21:13
  • exactly. The problem is that setup.py does not have any available option to install a wheel package. The only way I can see is what I suggested you to do. The other ways are more hacky ways and I would not recommend you to do so. – Gal Ben David Feb 27 '16 at 21:22
  • Can you add more info for the third? Thanks. – panda0 Jan 07 '18 at 14:32
  • 8
    this is a terrible accepted answer. it has no details on the solutions – jmerkow Sep 27 '18 at 17:43