11

The following command installs a Python package from a git repository:

$ pip install git+ssh://git@github.com/username/repo.git
Collecting git+ssh://git@github.com/username/repo.git
  Cloning ssh://git@github.com/username/repo.git to /tmp/pip-req-build-8s4nci15

I'm not 100% certain, but as it takes quite long I guess it clones every commit. But for installation, I only need the latest.

Instead, I would like a shallow clone (with --depth 1). Is that possible?

Martin Thoma
  • 124,992
  • 159
  • 614
  • 958
  • 1
    What is time consuming is not structure of the project with its folder, but the number of commits. You also have the option to use the `.zip` URL for downlaoding the repo. – Klaus D. Oct 25 '18 at 12:34
  • [BitBucket - download source as ZIP](https://stackoverflow.com/q/13044749/562769) – Martin Thoma Oct 25 '18 at 12:45
  • Hm... how do I do that for private repositories? It works through SSH keys with the solution above. Is there something similar for the zip files? – Martin Thoma Oct 25 '18 at 12:48
  • You can try to use an private access token: `https://username:token@github…` or without username. – Klaus D. Oct 25 '18 at 12:51
  • @MartinThoma, it likely depends on where the archives are hosted. Are you specifically talking about GitHub? Can you briefly describe your hosting environment (e.g. is it a VPS, are you using something like Heroku, etc.)? Might something like [Gemfury](https://gemfury.com/) help? – ChrisGPT was on strike Oct 25 '18 at 12:53
  • Currently, I use bitbucket. – Martin Thoma Oct 25 '18 at 13:45
  • Currently, I use bitbucket for the git code and a Docker container in a private cloud. – Martin Thoma Oct 25 '18 at 14:00

2 Answers2

12

Adding --depth 1 to the git clone command has been discussed at length and, for the time being, rejected by pip's maintainers. It appears to be a more complex issue than one might expect, particularly since it would break setuptools_scm which is widely used.

As Klaus said in the comments you can avoid the overhead of git clone by pointing to a remote archive file instead of using a Git URL:

pip install http://my.package.repo/SomePackage-1.0.4.zip
ChrisGPT was on strike
  • 127,765
  • 105
  • 273
  • 257
  • 1
    Hmm... read over the linked threads. I guess I'm missing something, because it face value it looks as though the PIP maintainers were going out of their way to avoid finding a working compromise, and IMO that could backfire on their stated goals spectacularly... – Mark Adelsberger Oct 25 '18 at 13:32
  • I'm not involved in the decision and I don't have enough context but I agree that there is probably a good solution. For now it looks like one hasn't been implemented in `pip`. – ChrisGPT was on strike Oct 25 '18 at 13:54
1

In addition to work-arounds (like ZIP) that might be available depending on hosting, a generic git work-around would be to pre-create the shallow clone, and then (since your clone is itself a git repo) point PIP to the clone.

This is not ideal (hence "work-around") in that, for an automated case, you would have to script creation of the shallow clone(s) before invoking PIP, and the data you give to PIP would list the clones' URL's instead of the canonical URL's for the respective packages.

Also, for the reasons called out in the PIP 'depth=1' debate, it might end up not working for you depending on what tools you (or your dependencies) use. It seems git describe is the crux of the issue (or at least, of one common issue); so you might be able to put a tag on the one commit you hold locally as a further work-around.

Mark Adelsberger
  • 42,148
  • 4
  • 35
  • 52