3

Wasn't able to find another question quite like this, but let me know if there is one which covers all the same elements (remote access to host machine, Docker container set up there, Github 2FA to access private repos).

I recently joined a company where I'm remotely ssh-ing into the host machines from my laptop at home. On those machines my colleagues each set up a Docker container and I did the same (my first time).

Having said that, I use the following workflows to (a) push to my company's private Github repos (let's say, https://github.com/my_company_name/my_company_repo), (b) clone and install from my company's private Github repos (let's say, https://github.com/my_company_name/colleague_repo).

For (a) I first navigate to my terminal inside the Docker container, then to the repo directory, git fetch from my repo and then (after adding/comitting etc) git push to it, at which point I have to fill in my username and password. Password doesn't work here; I have to fill in my personal access token (which I created with read & write permissions).

For (b) I first git clone from the colleague's repo, and have to enter my username and password. (Again, personal access token required instead of password.) Then I pip install -e to install the repo by name.

I would like to avoid having to constantly supply my credentials, by using some kind of appropriate configuration in my Dockerfile. (So for (a) I would like to just pop open my terminal and git push <origin> <master> and that's it.) I would also like to clone and install my colleague's repos in the Dockerfile itself (i.e. do all the cloning and installing business in the Docker build), since there is a pretty specific set of company repos to be installed everytime -- but as you can imagine, the authentication isn't working.

I tried to add lines like

RUN git config --global user.name <my_username>
RUN git config --global user.password <personal_access_token>
RUN pip install -e git+https://github.com/my_company_name/colleague_repo

to my Dockerfile (and have also tried my actual password in the second line). Didn't work -- got the same message that authentication had failed.

Anyone able to help?

Mobeus Zoom
  • 598
  • 5
  • 19
  • 1
    `user.password` is never a git config key. Have you tried `https://:@github.com/blah_blah`? – iBug Mar 11 '21 at 19:35
  • Note that you need to use the PAT instead of the password. Anyways, you could also use SSH for authentication. – dan1st Mar 11 '21 at 19:37
  • @iBug please elaborate: where do I try putting that in the Dockerfile? If in the installs, how does it help me handle pushes? Is `````` the PAT? And isn't there a concern about this being publicly available (to my colleagues) in the Dockerfile on the host machine? – Mobeus Zoom Mar 11 '21 at 21:52
  • @dan1st please elaborate - how would I go about this? And as with the HTTPS method, is there a concern about the ssh private key then being publicly visible (to my colleagues) along with(?) the Dockerfile? – Mobeus Zoom Mar 11 '21 at 21:54
  • Instead of a password, you can also [upload an SSH key to GitHub and use that](https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh). After setting this up, the SSH key will be permitted to pull/push from/to every repository you have access to. The SSH key needs to be kept private as it can be used to access GitHub with your user. – dan1st Mar 11 '21 at 21:58
  • If you don't want to type your password/token every time without saving it on the remote device, you can create a macro that sends username/PAT over the ssh session. – dan1st Mar 11 '21 at 21:59
  • @dan1st if you want to post an answer showing how you'd configure a Docker file with that, and also how to keep the SSH key private (if it'll have to be on the company machine?), I'd appreciate it. I believe there are ways like ```libsecret``` to keep Git credentials secret (for example, https://www.softwaredeveloper.blog/git-credential-storage-libsecret with HTTPS authentication) but I'm not sure how to put that in my Dockerfile – Mobeus Zoom Mar 11 '21 at 22:05
  • What do you mean with private? Not in the Dockerfile? Not in the host machine running the containers? – dan1st Mar 11 '21 at 22:07
  • @dan1st Both, I suppose? I guess anything on the host machine is ultimately readable to my colleagues? – Mobeus Zoom Mar 11 '21 at 22:13
  • You need to make a macro as you cannot store anything securely, then. An SSH key would not help you in that case. – dan1st Mar 11 '21 at 22:15

1 Answers1

1

You could use an SSH key, with a multi-stage approach as illustrated in "Access Private Repositories from Your Dockerfile Without Leaving Behind Your SSH Keys" by Vladislav Supalov

# this is our first build stage, it will not persist in the final image
FROM ubuntu as intermediate

# install git
RUN apt-get update
RUN apt-get install -y git

# add credentials on build
ARG SSH_PRIVATE_KEY
RUN mkdir /root/.ssh/
RUN echo "${SSH_PRIVATE_KEY}" > /root/.ssh/id_rsa

# make sure your domain is accepted
RUN touch /root/.ssh/known_hosts
RUN ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts

RUN git clone git@bitbucket.org:your-user/your-repo.git

FROM ubuntu
# copy the repository form the previous image
COPY --from=intermediate /your-repo /srv/your-repo
# ... actually use the repo :)

The more modern approach is by using BuildKit

With the new SSH mount type you can allow your Docker build to make use of your host’s SSH keys.

Here’s how it looks like:

RUN --mount=type=ssh ...

You add the new mount type to your RUN command, and the whole process is taken care of for you.

See BuildKit / Dockerfile frontend syntaxes/ RUN --mount=type=ssh

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • 1
    thanks. There are a few details that seem to be missing from the tutorial though. With the SSH method in the intermediate build: 1. How exactly do I modify ```RUN ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts``` and ```RUN git clone git@bitbucket.org:your-user/your-repo.git``` for my purpose? (accessing Github) 2. What follows ```COPY --from=intermediate /your-repo /srv/your-repo```? My exact Dockerfile as it currently is? If so, at what stage and how do we delete the intermediate image? – Mobeus Zoom Mar 12 '21 at 19:17
  • 1
    and with the BuildKit, really there's not much there to go off especially for a beginner like me. For example, 1. What's going on in ```mkdir -p -m 0700 ~/.ssh```? Or in ```ssh -q -T git@gitlab.com 2>&1 | tee /hello``` after specifying mount-type? 2. How do I modify this for accessing Github (rather than Gitlab) and especially, if I don't have ```apk``` because I am not use alpine (rather python3)? If you could draw up an example working with the conditions I set out in the OP that would be amazing – Mobeus Zoom Mar 12 '21 at 19:22
  • @MobeusZoom 1. Replace bitbucket by the relevant remote hosting service for you (github.com, I presume?) Replace the SSH URL of the repo by yours. 2. Yes. No need to delete the intermediate image. The point is: the final image does not include the intermediate one, which means it does not include any sensitive data, like your private SSH key. – VonC Mar 12 '21 at 19:22
  • 1
    what build command would I run then? Does it need to have ```--squash```? Do I use ```docker build``` twice or just once? I'm a bit confused – Mobeus Zoom Mar 12 '21 at 19:32
  • @MobeusZoom A normal docker build (for the multi-stage approach). No squash necessary. – VonC Mar 12 '21 at 19:34
  • Ok thanks I'll try it. If you think there is any advantage to the BuildKit approach I'd welcome some clarifications/detail on it – Mobeus Zoom Mar 12 '21 at 19:35
  • While the overall attempt to assist is appreciated, I echo Mobeus' confusions. Think this answer/comments need more detail to be effective. – Ruben Flam-Shepherd Dec 26 '21 at 00:18