How to securely git clone/pip install a private repository into my docker image?

Question

I have a private repo that contains packages I want to pip install. I've spent quite a bit of time reading over various forums and articles about different ways to securely do this. There doesn't seem to be a a consensus on how to best do this (if at all possible). I obviously don't want to expose any ssh keys/secrets in my dockerfile -- I want to be careful about making them available via docker history.

Answer 1

As explained in "Securely build small python docker image from private git repos", you would need to use, with Docker 18.09+

• --ssh
  You can use the --ssh flag to forward your existing SSH agent key to the builder. Instead of transferring the key data, docker will just notify the builder that such capability is available.
  Now when the builder needs access to a remote server through SSH, it will dial back to the client and ask it to sign the specific request needed for this connection.
  The key itself never leaves the client, and as soon as the command that requested the access has completed there is no information on the builder side to reestablish that remote connection later.

• Secrets:
  Provides a mount option during the build at /var/run/secrets available only for the command that used it and is not included in the created layer.

That is:

docker build --ssh github_ssh_key=/path/to/.ssh/git_ssh_id_rsa .

only the agent connection is shared with that command, and not the actual content of the private key.
no other commands/steps in the Dockerfile will have access to it.

The Dockerfile, in a multistage first step, would give a key name github_ssh_key so we can use it when we invoke docker build:

RUN --mount=type=ssh,id=github_ssh_key pip wheel \
    --no-cache \
    --requirement requirements.txt \
--wheel-dir=/app/wheels

---

The OP Jesus Garcia did report (in the comments) making it work:

I had to use 2 separate RUN commands.

I'm not sure if it's a limitation of this new feature, or the way I was trying to string together multiple commands in my RUN but I kept getting a publickey permission denied error when I added it as other commands && /bin/sh -c "mount=type=ssh,id=github_ssh_key pip install private-repo" vs RUN --mount=type=ssh,id=github_ssh_key pip install private-repo && more commands ...

Answer 2

This worked for me with docker 19.03.5 (but should work with 18.09+):

Say your private repo is on Github, github.com/user/repo.git and you want to pip install it as part of a docker build. The ssh key to the repo is located on the host at ~/.ssh/my_key.

In the Dockerfile:

RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh pip install git+ssh://git@github.com/user/repo.git

Then, on the host:

export DOCKER_BUILDKIT=1
eval `ssh-agent`
ssh-add ~/.ssh/my_key
docker build --ssh default=$SSH_AUTH_SOCK .

Answer 3

Despite using modern (19.03.8) Docker, neither of the above solutions worked for me (even after prepending DOCKER_BUILDKIT=1) without an extra line I found in Alexandra Ulsh's blog post, which I think is significant enough to deserve an extra answer rather than being buried in the comments.

Though the --secret flag appears in a production version of Docker, there are a few hints that support is still somewhat experimental.

For example, the first line of a Dockerfile using build secrets must be # syntax = docker/dockerfile:1.0-experimental. Without this line you’ll get the error failed to create LLB definition: Dockerfile parse error line 6: Unknown flag: mount. This line enables the Docker CLI to use the “experimental Dockerfile frontend” for Moby BuildKit.

TL;DR add the commment # syntax = docker/dockerfile:1.0-experimental at the top of your Dockerfile, then follow Dave Rejkher's answer.

Answer 4

I am on Docker version 19.03.13 (build 4484c46d9d), and using the --ssh feature as described in VonC's and Dave Reikher's answers did not work for me. I still keep getting the error that:

ERROR: Command errored out with exit status 128: git clone -q 'ssh://git@hostname:port/group/repository.git' /tmp/pip-install-8eeeaipo/baseapi Check the logs for full command output.

Note that I am not pip install-ing from a Github repo, but from some really private/internal repository. The examples also from the Docker docs on --ssh also show a "github.com" repo URL. I am not sure if that's the case (it shouldn't be), but I've traced it up to pip/git not recognizing the hostname of the URL, while if using "github.com" it works as advertised.

I made it work instead by using just the plain --secret function (still part of BuildKit), by passing-in the SSH key (~/.ssh/id_rsa) and the ~/.ssh/known_hosts file as secrets.

Dockerfile

# syntax=docker/dockerfile:1.0-experimental

FROM python:3.8-alpine3.11

RUN apk add openssh-client git
RUN mkdir -p -m 0600 /root/.ssh
RUN --mount=type=secret,id=known_hosts,dst=/root/.ssh/known_hosts \
    --mount=type=secret,id=ssh_key,dst=/root/.ssh/id_rsa \
    ssh-keyscan -H -t rsa hostname.of.private.repo \
    && pip install --no-cache-dir git+ssh://git@hostname:port/group/repository.git@12345678#egg=baseapi

Build Command

DOCKER_BUILDKIT=1 docker build \
    --secret id=known_hosts,src=~/.ssh/known_hosts \
    --secret id=ssh_key,src=~/.ssh/id_rsa \
    -t name:tag \
    -f Dockerfile \
    .

I checked the ~/.ssh folder on the built image and it seems it's still correct that it does not persist the SSH keys. The mounted files remain but they are empty:

/app # ls -l ~/.ssh/
total 0
-rwxr-xr-x    1 root     root             0 Nov 19 11:29 id_rsa
-rwxr-xr-x    1 root     root             0 Nov 19 11:29 known_hosts
/app # cat ~/.ssh/id_rsa 
/app # cat ~/.ssh/known_hosts

And trying to pip install the same private package should fail.

For reference:

* Docker: 19.03.13 (build 4484c46d9d)
* Image
    - pip 20.2.4 from /usr/local/lib/python3.8/site-packages/pip (python 3.8)
    - git 2.22.4
    - OpenSSH_8.1p1, OpenSSL 1.1.1g  21 Apr 2020

Answer 5

Use RUN --mount=type=ssh

ci.yml

build_docker:
  stage: build
  image: docker:latest
  variables:
    DOCKER_HOST: tcp://docker:2375
    # This instructs Docker not to start over TLS.
    DOCKER_TLS_CERTDIR: ""

  before_script:
    - apk add --no-cache curl jq python3 py3-pip git bind-tools openssh-client
    - echo $DOCKERPASS | docker login --username $DOCKERUSER --password-stdin
    - mkdir -p -m 0700 ~/.ssh
    - ssh-keyscan private-url >> ~/.ssh/known_hosts 
    - chmod 600 $SSH_KEY

  services:
    - docker:dind

  script:
    - export DOCKER_BUILDKIT=1
    - eval $(ssh-agent)
    - ssh-add $SSH_KEY
    - docker build -t tag_name  --ssh default=$SSH_AUTH_SOCK .

dockerfile

FROM puckel/docker-airflow:1.10.1

ARG SSH_PRIVATE_KEY

USER root
RUN apt-get update && apt-get install -y git openssh-client


RUN ssh-keyscan private-url >> ~/.ssh/known_hosts

COPY requirements.txt .

RUN --mount=type=ssh pip install git+ssh://private-url/project/repo.git
RUN pip install -r requirements.txt

see the picture https://feiyang233.club/img/vm/23.png