1

I am storing distributable wheel packages from several modules in a private GitHub repository. Here is the repository structure:

myrepo\
       | module1\
       |         | pyproject.toml
       |         | src\
       |         |     | *.py
       |         | dist\
       |                | module1.whl
       | module2\
                 | pyproject.toml
                 | src\
                 |     | *.py
                 | dist\
                        | module2.whl

Currently, I'm installing them using the pip tool from the source, which works. But, each time it clones the whole repository and builds the package locally.

pip install ssh+git://git@github.com/myorg/myrepo.git#subdirectory=module1

Is there a way to tell pip to only download and install the module1.whl and module2.whl files and not the whole repository?

I've tried the below approaches so far, but none of them works:

pip install ssh+git://git@github.com/myorg/myrepo.git#subdirectory=module1/dist/module1-0.1.0-py3-none-any.whl

and

pip install https://github.com/myorg/myrepo/blob/main/module1/dist/module1-0.1.0-py3-none-any.whl

I've already checked existing questions like Is it possible to use pip to install a package from a private github repository? and Distributing my python module inside organization, but none of them describe this use case.

Edited

This is already possible for Public repositories like, so it is not something strange:

pip install https://github.com/numpy/numpy/releases/download/vx.y.z/numpy-x.y.z.tar.gz
PyGuy
  • 434
  • 5
  • 15
  • I usually use `python -m pip install -e .` when inside a similar `module1 ` for example Edit: From within the repo you can try `python -m pip install -e module1` – rafidini Dec 09 '22 at 21:33
  • I am already doing that for local development, but it is not useful for distributing across the whole organization or using in the CI/CD build systems. – PyGuy Dec 09 '22 at 21:39
  • You can download the wheel and then run pip install on the .whl file itself: `pip install path/to/module.whl` See https://wheel.readthedocs.io/en/stable/user_guide.html#installing-wheels – joanis Dec 09 '22 at 21:39
  • Combine that with an operation that downloads the wheel files and you're set. – joanis Dec 09 '22 at 21:40
  • Distribute *source code* from the Git repository. Distribute wheels from a private PyPi repository. – chepner Dec 09 '22 at 21:40
  • 1
    I was going to suggest making the wheels assets in your releases, but uploading the wheels to a private PyPI repo seems even better. Downloading assets from a github release is very simple to automate. – joanis Dec 09 '22 at 21:42
  • @joanis I would love to know how that is possible for a **private** repository. I've already used it for **public** repositories. – PyGuy Dec 09 '22 at 21:47
  • You need to use your PAT on the curl command, and then it should work. See https://stackoverflow.com/a/35280061/3216427 – joanis Dec 09 '22 at 21:52

1 Answers1

1

After elaborate exploration, I found that it is not possible to install release packages of a private GitHub repository by pip.

The reason is that GitHub only allows downloading such files using GitHub tokens and not by deploy keys.

Suggested ways to download the file and add it to the container, or to use API tokens to download the package and install it in the container and install it from there are not viable as well. Because you either have to update the file manually, or update the API token.

The only possible way is to create a private package repository on google and push the release packages there.

PyGuy
  • 434
  • 5
  • 15