1

I am building a project which requires some dependencies on private python repositories. This repo has inside a submodule which also points out to a private repo. I can fetch it on my machine using an ssh key which is not convenient inside a container.

In my dockerfile, I am blocking on this step:

RUN pip3 install wheel \
  && pip3 wheel git+https://${GITHUB_TOKEN}@github.com/owner/repo.git@master \
--wheel-dir=/svc/wheels

It can fetch repo without problem but cannot handle the submodule. I guess pip resolves the URL and doesn't pass the token initially transferred.

I don't really know how to workaround the issue. I could have the folder locally and copy it, or setup a private pip repo I'd access during build time (if it's possible), or use ssh keys inside the container (sounds like a terrible approach to me).

Is there any "standard way to do it" ? If not what would you suggest?

EDIT1: this seems like a git issue as git clone --recursive doesn't work neither

Dharman
  • 30,962
  • 25
  • 85
  • 135
David Bensoussan
  • 2,887
  • 2
  • 38
  • 55

2 Answers2

1

Rather than passing ${GITHUB_TOKEN) on the command line, you should be able to configure that into a .netrc file as described e.g. in this answer.

Your Dockerfile would include something like:

RUN echo machine github.com login ${GITHUB_TOKEN} > ~/.netrc
RUN pip3 install wheel \
  && pip3 wheel git+https://github.com/owner/repo.git@master \
--wheel-dir=/svc/wheels
larsks
  • 277,717
  • 41
  • 399
  • 399
  • I didn't know about this feature, thanks for the advice. nevertheless, it still gives the same result with a private submodule inside – David Bensoussan Dec 14 '18 at 15:52
  • 1
    Odd, it works just fine my test, although I'm using `git` directly rather than `pip` because I have access to a limited number of private github repositories and none of them are hosting a Python module. Did the `git clone` for the master repository succeed? – larsks Dec 14 '18 at 16:20
  • I just found out the issue in my submodule declaration. It was set up using ssh. `url = git@github.com:` instead `url = https://github.com/`. Thanks for spending time and effort testing it. – David Bensoussan Dec 14 '18 at 16:23
1

Use the following command in the dockerfile, before RUN pip install ...

RUN git config --global url."https://${GITHUB_TOKEN}@github".insteadOf https://github

This will replace the occurrences of https://github with https://${GITHUB_TOKEN}@github, even at a submodule level when git command is executed.

Note: In the parent repository, the submodule url (defined in .gitmodules) should be set to https://github instead of git@github.

Pushkar Kumar Jain
  • 11
  • 2