6

Background

I have two python projects. Project A and Project B.

Each project has its own virtual environment and its own setup.py.

Both projects are not pure py files and has "build" process like building extensions, generate source etc.

A is dependent on B (setup.py install_requires points on B).

Each project is built and published/distributed as a wheel to pypi on-premise repository.

pip install w/wo -e (development mode) of project A, nicely installs project B into project A venv site-packages.

So far everything is working nicely and "by the book".

Now, my story get complicated ...

I would like to develop the two projects together without having to publish B in order A to consume it.

As example, I would like to:

  1. Change something in B.
  2. build B (setup.py build).
  3. NOT publish B as wheel to pypi.
  4. Goal - Project A will "know" the modified Project B. WITHOUT manually tweaking sys.path.

I actually want the pip install -e of project A to install project B in development mode too.

After some reading I understood (hopefully correctly) that one can define a "distribution" as a local source folder by --find-links flag.

I defined project B root folder in --find-links.

I tried it with (on project A setup.py folder):

  1. pip install . -e --find-links = file:///path/to/B

  2. pip install . -e --find-links = git+file://path/to/B

Both did not work.

BTW, Puting in the path to B wheel or egg of B,

ex: pip install . -e --find-links = file:///path/to/B.whl

did work but this is not what I am looking for.

Hope you are still with me :-) (sorry for the tedious story)

What am I missing?

Tx

Lior Cohen
  • 5,570
  • 2
  • 14
  • 30
  • 1
    [this question](https://stackoverflow.com/questions/20101834/pip-install-from-git-repo-branch) might help you... try [this](https://stackoverflow.com/a/46666538/7454638) `pip install git+https://github.com/user/project.git@version` or, if slow, [try installing from zip](https://stackoverflow.com/a/24811490/7454638) You can also have different files for development and production dependencies, installing them recursively – leoschet Jun 20 '18 at 22:29
  • `--find-links` makes `pip` to look into either a directory with tars/eggs/wheels or an HTML file with links to packages in downloadable forms. You cannot just point it to a local project. – phd Jun 20 '18 at 23:04
  • Leoschet - this does not answer my usecase. My dependent is local folder and not remote or zip. – Lior Cohen Jun 20 '18 at 23:12
  • @Phd - so no other way to resolve it but writing "out of pip" scripts to pip -e the dependencies too or to add b to sys.path? – Lior Cohen Jun 20 '18 at 23:15
  • It's not quite clear to me why do you want to pass the dependency via `find-links`. Won't `pip install --editable path/to/B --editable path/to/A` already suffice? – hoefling Jun 22 '18 at 14:37
  • @hoefling - your suggestion pass the management to the user and outside of pip config. What I would like is to consume B as any other "install_requires" of A but declared it as locally and editable (if found locally). I probably will have to write script that will hook to pip install -e path/to/A. But still looking for "in pip" solution. – Lior Cohen Jun 22 '18 at 21:03
  • You can't declare an editable dependency in setup script. You can pass a path to the dependency source files, e.g. via `dependency_links` or `pip`'s `find-links`, but it will install the dependency in normal, non-editable mode. – hoefling Jun 22 '18 at 21:12
  • 1
    @hoefling - Thanks. I wish there was a way. BTW, I did not managed to declare dependency to the local source files at all. I tried to point find-links to the source setup.py folder and it just complained it can not find a distribution there. All the examples I see online are git+http. Do you know a working example that does this? – Lior Cohen Jun 22 '18 at 21:21
  • I'm also quite interested on this sub ject: how would it be done when you have a tar.gz already generated? – F3RD3F Oct 01 '18 at 13:50

1 Answers1

1

Let me try and restate the problem:

  • You work on two python packages, package A and package B. A depends on B.
  • You have chosen to develop these two packages in separate virtual environments.
  • You would like to make some local changes to package B and have package A use the modified version of package B.

I apologize if I'm missing something here, but why not simply install both packages in the same virtual environment, which will make this problem go away?

I.e. after creating your environment, you install package B in editable mode, followed by installing package A. Any changes in B will be picked up by A, no changes needed.

leopold.talirz
  • 640
  • 7
  • 19