749

I've installed a library using the command

pip install git+git://github.com/mozilla/elasticutils.git

which installs it directly from a Github repository. This works fine and I want to have that dependency in my requirements.txt. I've looked at other tickets like this but that didn't solve my problem. If I put something like

-f git+git://github.com/mozilla/elasticutils.git
elasticutils==0.7.dev

in the requirements.txt file, a pip install -r requirements.txt results in the following output:

Downloading/unpacking elasticutils==0.7.dev (from -r requirements.txt (line 20))
  Could not find a version that satisfies the requirement elasticutils==0.7.dev (from -r requirements.txt (line 20)) (from versions: )
No distributions matching the version for elasticutils==0.7.dev (from -r requirements.txt (line 20))

The documentation of the requirements file does not mention links using the git+git protocol specifier, so maybe this is just not supported.

Does anybody have a solution for my problem?

Danny Varod
  • 17,324
  • 5
  • 69
  • 111
Alfe
  • 56,346
  • 20
  • 107
  • 159

10 Answers10

924

Normally your requirements.txt file would look something like this:

package-one==1.9.4
package-two==3.7.1
package-three==1.0.1
...

To specify a Github repo, you do not need the package-name== convention.

The examples below update package-two using a GitHub repo. The text after @ denotes the specifics of the package.

Specify commit hash (41b95ec in the context of updated requirements.txt):

package-one==1.9.4
package-two @ git+https://github.com/owner/repo@41b95ec
package-three==1.0.1

Specify branch name (main):

package-two @ git+https://github.com/owner/repo@main

Specify tag (0.1):

package-two @ git+https://github.com/owner/repo@0.1

Specify release (3.7.1):

package-two @ git+https://github.com/owner/repo@releases/tag/v3.7.1

Note that in certain versions of pip you will need to update the package version in the package's setup.py, or pip will assume the requirement is already satisfied and not install the new version. For instance, if you have 1.2.1 installed, and want to fork this package with your own version, you could use the above technique in your requirements.txt and then update setup.py to 1.2.1.1.

See also the pip documentation on VCS support.

YPCrumble
  • 26,610
  • 23
  • 107
  • 172
  • 139
    Out of all the other answers, I can't believe *none* of them just showed a requirements file with a blend of "normal" reqs with a git one thrown in for comparison. I was so thrown off by what looked like command-line (`-e`) options. Thanks for showing a blend of both so I could put this in context! – Hendy Oct 22 '17 at 19:47
  • 5
    Pointing to the release `3.7.1` with `git+git://github.com/path/to/package-two@releases/tag/v3.7.1#egg=package-two` did not work for me. What worked for me was `git+git://github.com/path/to/package-two@3.7.1#egg=package-two`. – Jean Paul Mar 04 '20 at 13:20
  • 13
    This answer was very helpful. One thing though. Those `git+git://...` notation somehow caused ssh-relative errors on my Linux box. So I ended up switching them to `git+https://...` notation and then they work perfectly. – RayLuo Mar 28 '20 at 02:49
  • 2
    I wasn't sure if you should what to put for `egg=`. I forked a project where the package name that you `pip install` has a dash in it (`package-two`), but the module you import has an underscore in it (`package_two`). I used the name of the imported module (with the underscore) and it works fine. – ChrisCrossCrash Apr 15 '21 at 18:31
  • 9
    [GitHub dropped support](https://github.blog/2021-09-01-improving-git-protocol-security-github/) for the git protocol -- you'll have to use https. – MikeTwo Nov 02 '21 at 14:17
  • 1
    @MikeTwo updated - does the new version look correct to you? – YPCrumble Jan 12 '22 at 13:58
  • Excelent answere. It helps me a lot! – Lucas Brito Jul 15 '22 at 16:50
  • 2
    I think you may also need `setup.py` or `pyproject.toml`. – wsams Oct 14 '22 at 22:07
  • What if I want to install something that isn't in the root of a repo? Like if a repo has many folders with many packages how can I install any individual package? – Cruncher Nov 04 '22 at 17:22
  • @wsams you are correct that for certain recent versions of pip you will need to update `setup.py` - updated the answer. – YPCrumble Apr 09 '23 at 17:58
  • Does `#egg=package-two` still work in 2023? I had to use `package-two @ git+https:...` instead to get it working for my project, specifically in GitHub Actions. – someone Apr 13 '23 at 18:50
  • @someone I think you're correct, updating the answer for latest pip versions. – YPCrumble Apr 14 '23 at 22:53
  • @YPCrumble I suggest you add an example where the root of the Python project is in a sub-directory of the git repository. – sinoroc Apr 15 '23 at 15:45
390

“Editable” packages syntax can be used in requirements.txt to import packages from a variety of VCS (git, hg, bzr, svn):

-e git://github.com/mozilla/elasticutils.git#egg=elasticutils

Also, it is possible to point to particular commit:

-e git://github.com/mozilla/elasticutils.git@000b14389171a9f0d7d713466b32bc649b0bed8e#egg=elasticutils
ArtOfWarfare
  • 20,617
  • 19
  • 137
  • 193
stalk
  • 11,934
  • 4
  • 36
  • 58
  • 4
    I didn't manage to checkout locally such an editable version (due to syntax problems, probably) and so ended up using the `git+git` variant (which worked). In the `requirements.txt` your version works, so thank you very much :) – Alfe May 16 '13 at 10:42
  • 1
    The `docs` link was not working for me; I've used [an older one](http://www.pip-installer.org/en/1.1/requirements.html). – Sergey Orshanskiy Dec 09 '13 at 00:11
  • 51
    What I did not understand is that the syntax showed is exactly what goes in requirements, i.e. there is no package name before the -e. – sage Dec 22 '13 at 01:57
  • 6
    Adding "-e" isn't necessary depending on whether you want the package to be in editable mode, see answer by @qff . – Tianwei Chen Feb 23 '16 at 16:25
  • In OpenShift Online Next Gen Developer Preview, build throws error: `AssertionError: Sorry, 'git://github.com/Username/repo-name.git#egg=repo-name' is a malformed VCS url. The format is +://, e.g. svn+http://myrepo/svn/MyApp#egg=MyApp` – user1063287 Jul 11 '16 at 12:48
  • What to do in case of using --src option in requirements.txt? I am trying: `--src ./myvenv/lib/python2.7/site-packages -e git+https://github.com/Remak-a-s/rmkGoogleAPIs.git#egg=rmkgoogleapi` and it said: `pip: error: no such option: --src` – zhukovgreen Oct 20 '16 at 09:20
  • 17
    Shouldn't it be **`-e git+git://`** instead of `-e git://`? I got a *"should either be a path to a local project or a VCS url beginning with svn+, git+, hg+, or bzr+"* error message. – Srini Feb 23 '17 at 22:47
  • 2
    i get this `Could not detect requirement name, please specify one with #egg=` – abbood Jul 21 '17 at 06:27
  • @zhukovgreen you should include `--src path/to/env/site-packages` in command: `pip install -r requirements.txt --src ...` – Wassadamo Oct 02 '20 at 22:03
  • Looks like this will be deprecated soon: https://github.com/pypa/pip/issues/7554. Looks like the correct way to do this is now: -e git+ssh://git@github.com:path/to/your/repo (use https or http (insecure) if you don't have ssh set up) – apteryxlabs Jan 19 '21 at 15:58
  • 1
    Why this random chunk of code without context is upvoted and accepted I can't understand. – EugZol Feb 07 '21 at 14:41
  • What is the "-e" meaning? – Lucas Brito Jul 15 '22 at 16:51
209

requirements.txt allows the following ways of specifying a dependency on a package in a git repository as of pip 7.0:1

[-e] git+git://git.myproject.org/SomeProject#egg=SomeProject
[-e] git+https://git.myproject.org/SomeProject#egg=SomeProject
[-e] git+ssh://git.myproject.org/SomeProject#egg=SomeProject
-e git+git@git.myproject.org:SomeProject#egg=SomeProject (deprecated as of Jan 2020)

For Github that means you can do (notice the omitted -e):

git+git://github.com/mozilla/elasticutils.git#egg=elasticutils

Why the extra answer?
I got somewhat confused by the -e flag in the other answers so here's my clarification:

The -e or --editable flag means that the package is installed in <venv path>/src/SomeProject and thus not in the deeply buried <venv path>/lib/pythonX.X/site-packages/SomeProject it would otherwise be placed in.2

Documentation

qff
  • 5,524
  • 3
  • 37
  • 62
  • 14
    But note that if you omit the `-e` your next `pip freeze` may not give the correct results for this package – Maccesch Jun 21 '17 at 13:05
  • 5
    Note: `git+git@` urls [are deprecated](https://github.com/pypa/pip/issues/7554) since [Jan 2020](https://github.com/pypa/pip/pull/7543). The others are still supported. – idbrii Dec 16 '20 at 02:45
98

First, install with git+git or git+https, in any way you know. Example of installing kronok's branch of the brabeion project:

pip install -e git+https://github.com/kronok/brabeion.git@12efe6aa06b85ae5ff725d3033e38f624e0a616f#egg=brabeion

Second, use pip freeze > requirements.txt to get the right thing in your requirements.txt. In this case, you will get

-e git+https://github.com/kronok/brabeion.git@12efe6aa06b85ae5ff725d3033e38f624e0a616f#egg=brabeion-master

Third, test the result:

pip uninstall brabeion
pip install -r requirements.txt
Sergey Orshanskiy
  • 6,794
  • 1
  • 46
  • 50
25

Since pip v1.5, (released Jan 1 2014: CHANGELOG, PR) you may also specify a subdirectory of a git repo to contain your module. The syntax looks like this:

pip install -e git+https://git.repo/some_repo.git#egg=my_subdir_pkg&subdirectory=my_subdir_pkg # install a python package from a repo subdirectory

Note: As a pip module author, ideally you'd probably want to publish your module in it's own top-level repo if you can. Yet this feature is helpful for some pre-existing repos that contain python modules in subdirectories. You might be forced to install them this way if they are not published to pypi too.

TrinitronX
  • 4,959
  • 3
  • 39
  • 66
16

Github has zip endpoints that in my opinion are preferable to using the git protocol. The advantages are:

  • You don't have to specify #egg=<project name>
  • Git doesn't need to be installed in your environment, which is nice for containerized environments
  • It works much better with pip hashing and caching
  • The URL structure is easier to remember and more discoverable

You usually want requirements.txt entries to look like this, e.g. without the -e prefix:

https://github.com/org/package/archive/1a58aa586efd4bca37f2cfb9d9348958986aab6c.tar.gz

To install from main branch:

https://github.com/org/package/archive/main.tar.gz

There is also an equivalent .zip endpoint, but it was reported in a comment that always using the .tar.gz endpoint avoids problems with unicode package names.

antonagestam
  • 4,532
  • 3
  • 32
  • 44
  • 4
    As mentioned in a comment of this similar [SO answer](https://stackoverflow.com/a/24811490/943773), the zip extractor can have issues with unicode package names. Specifying `.tar.gz` instead of `.zip` will fix this. – ryanjdillon Sep 23 '21 at 10:20
  • 1
    @ryanjdillon I updated the answer to recommend `.tar.gz` instead of `.zip`. – antonagestam Oct 12 '21 at 08:41
  • This is the one answer I was looking for, thanks. – ramnes Feb 15 '22 at 19:14
14

None of these answers worked for me. The only thing that worked was:

git+https://github.com/path_to_my_project.git

No "e", no double "git" and no previous installs necessary.

Cybernetic
  • 12,628
  • 16
  • 93
  • 132
8

It seems like this is also a valid format:

gym-tictactoe @ git+https://github.com/haje01/gym-tictactoe.git@84e22fc28fe192ba0040bdd56a697f63d3d4a3d5

If you do a pip install "git+https://github.com/haje01/gym-tictactoe.git", then look at what got installed by running pip freeze, you will see the package described in this format and can copy and paste into requirements.txt.

Dustin Michels
  • 2,951
  • 2
  • 19
  • 31
  • This fixes my issue from the `private repo` and `setup.py` `install_requires` options. Thanks! – Nathaniel Varona Jan 28 '22 at 10:31
  • 1
    Just needed this today, and the solutions in older answers did not work for me, but this one worked well. I wonder if there was a change in pip? In particular, the `#egg=gym-tictactoe` at the end failed, but `gym-tictactoe @ ` at the beginning worked, with the same `git+https:...` URL either way. – someone Apr 13 '23 at 18:48
5

I'm finding that it's kind of tricky to get pip3 (v9.0.1, as installed by Ubuntu 18.04's package manager) to actually install the thing I tell it to install. I'm posting this answer to save anyone's time who runs into this problem.

Putting this into a requirements.txt file failed:

git+git://github.com/myname/myrepo.git@my-branch#egg=eggname

By "failed" I mean that while it downloaded the code from Git, it ended up installing the original version of the code, as found on PyPi, instead of the code in the repo on that branch.

However, installing the commmit instead of the branch name works:

git+git://github.com/myname/myrepo.git@d27d07c9e862feb939e56d0df19d5733ea7b4f4d#egg=eggname
Throw Away Account
  • 2,593
  • 18
  • 21
4

For private repositories, I found that these two work fine for me:

pip install https://${GITHUB_TOKEN}@github.com/owner/repo/archive/main.tar.gz

Where main.tar.gz refers to the main branch of your repo and can be replaced with other branch names. For more information and using the more recent Github API see here:

pip install https://${GITHUB_TOKEN}@api.github.com/repos/owner/repo/tarball/master

If you have git installed and available, then

pip install git+https://${GITHUB_TOKEN}@github.com/owner/repo.git@main

achieves the same, and it also allows for some more flexibility by appending @branch or @tag or @commit-hash. That approach, however, actually clones the repo into a local temp folder which can take a noticeable amount of time.

You can use the URLs in your requirements.txt, too.

Jens
  • 8,423
  • 9
  • 58
  • 78