28

Is it possible to specify (editable) source dependencies in setup.py that are known to reside on the local file system?

Consider the following directory structure, all of which lives in a single VCS repository:

projects
  utils
    setup.py
    ...
  app1
     setup.py
     ... # app1 files depend on ../utils
  app2
     setup.py
     ... # app2 files depend on ../utils

Given the following commands:

cd projects
mkvirtualenv app1
pip install -e app1

I'd like to have all the dependencies for app1 installed, including "utils", which is an "editable" dependency. Likewise, if I did the same for app2.

I've tried playing with all different combinations of file://... URLs in install_requires and dependency_links to no avail. I'd like to use a dependency link URL like src+file://../utils, which would tell setuptools that the source for the package is on the file system at this relative path. Is there a way to do this?

millerdev
  • 10,011
  • 2
  • 31
  • 27
  • This might be of some help: http://packages.python.org/distribute/setuptools.html#dependencies-that-aren-t-in-pypi – Rob Wouters Oct 19 '12 at 21:20
  • 1
    Thanks, Rob. I reviewed that page extensively before asking the question here. The page lists two categories of URLs: (1) direct download URLs, and (2) URLs of web pages that contain direct download links. I was hoping my issue would fall into the first category, but I can't figure out how to construct the URL. – millerdev Oct 22 '12 at 14:36
  • 5
    I submitted a [patch to pip](https://github.com/pypa/pip/pull/719) to support relative `file:` URLs in `dependency_links`, which solves my issue. Hopefully it gets merged soon. – millerdev Nov 08 '12 at 20:27
  • Just ran into this question, and the link Rob Wouters posted is broken. Here's one that works as of this comment: https://pythonhosted.org/setuptools/setuptools.html#dependencies-that-aren-t-in-pypi – Wisco crew Jul 11 '14 at 20:46
  • Why don't you create an account at #github and host your package/fork there. This would allow you refer to a specific commit of that repository as a dependency. Also, do you just want to get it to work on your localhost? Or do you want a robust solution that works for everyone interested in the package? –  Nov 19 '14 at 09:32
  • Package metadata is not the right place for this kind of information. For this there's requirements.txt where you can use local paths as shown in the Danver Braganza's answer. – Piotr Dobrogost May 22 '16 at 14:08
  • 1
    That link from Rob appears to be broken again @Wiscocrew. Here is the latest (I think): https://setuptools.pypa.io/en/latest/deprecated/dependency_links.html – ryanjdillon Dec 28 '22 at 20:14
  • 1
    I'm afraid I don't recall what exactly I linked 8 years ago, but that looks right @ryanjdillon – Wisco crew Dec 30 '22 at 16:57

3 Answers3

10

i managed to provide relative local dependency in setup.py with:

setup(
    install_requires=[
        'utils @ file://localhost/%s/../utils/' % os.getcwd().replace('\\', '/'),
        ],
    )

but maybe someone know better solution

Konstantin Maslyuk
  • 101
  • 1
  • 2
  • Took me forever to find this. So far this is the only working solution I've seen. Very useful for working with editable installed local packages. – waydegg Jan 30 '21 at 03:45
  • Same solution but simplified with f-strings for python 3.x. This also assumes the package you're referencing is _above_ one level in your file path hence the "..": ```python f"my_package @ file://localhost/{os.getcwd()}/../my_package/" ``` – waydegg Jan 30 '21 at 03:53
  • 1
    Useful, but only applies if installing via `develop` option, AKA `-e` flag. If you're installing the package via `pip install .`, you immediately lose your starting CWD. – Rorschach Mar 03 '22 at 10:01
  • @Rorschach `os.getcwd` works for me when I use `pip install .` (no `-e` flag) – Asker May 19 '23 at 23:30
7

I had an identical problem where I needed to depend on modules in a sibling folder. I was able to find a solution after stumbling upon https://caremad.io/2013/07/setup-vs-requirement/

I ended up requirements.txt to refer specifically to the file I wanted, and then installing everything with 

pip install -r requirements.txt

requirements.txt

-e ../utils                                                                                                                                                                    
-e .

And setup.py has all my other dependencies, including utils. When pip tries to install app1 itself, it realizes that the utils dependency has already been filled, and so passes over it, while installing the other requirements.

Danver Braganza
  • 1,295
  • 10
  • 10
  • the relative path in the requirements.txt work in case you entered the directory and executed the command...executing same command in the parent directory would not work. There are issues with relative path specification for pip requirements.txt file https://github.com/pypa/pip/issues/3772 – infinityLoop Oct 01 '16 at 21:50
0

When I want to work with a set of projects interrelated, I install all of them using /setup.py develop.

If I mistakenly or later I want to make a pip-installed module editable, I clone the source, and do a python setup.py develop on it too, substituting the existing one.

Just to get sure, I erase the reference in the virtualenv's site-packages and the package itself.

txomon
  • 642
  • 1
  • 6
  • 21