26

I'm trying to run a container that will expose a golang service from a package that I have on a private GitHub repo.

Since I am working with GCE, my starter image is google/debian:wheezy.

After installing all the required dependancies and tools, I am running 

RUN go get github.com/<my_org>/<my_package>

where the package is a private repo.

I have added my GitHub SSH keys to allow the cloning from the private repo to the docker file:

ADD priv/id_rsa /root/.ssh/id_rsa
ADD priv/id_rsa.pub /root/.ssh/id_rsa.pub

Still, I am getting an error during the go get process when go tried to clone the repo:

# cd .; git clone https://github.com/<my_org>/<my_package> /gopath/src/github.com/<my_org>/<my_package>
Cloning into '/gopath/src/github.com/<my_org>/<my_package>'...
fatal: could not read Username for 'https://github.com': No such device or address
package github.com/<my_org>/<my_package>: exit status 128

To debug the problem, from the Dockerfile, I am running:

RUN ssh-keyscan -t rsa github.com 2>&1 >> /root/.ssh/known_hosts

And this tells me there are some problems. It looks like validating the private key is OK but something weird is going on the the public key. This is the complete ssh-keyscan result:

OpenSSH_6.0p1 Debian-4+deb7u2, OpenSSL 1.0.1e 11 Feb 2013
Pseudo-terminal will not be allocated because stdin is not a terminal.
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to github.com [192.30.252.129] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type 1
debug1: Checking blacklist file /usr/share/ssh/blacklist.RSA-2048
debug1: Checking blacklist file /etc/ssh/blacklist.RSA-2048
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: Remote protocol version 2.0, remote software version libssh-0.6.0
debug1: no match: libssh-0.6.0
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.0p1 Debian-4+deb7u2
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-sha1 none
debug1: kex: client->server aes128-ctr hmac-sha1 none
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: RSA 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48
debug1: Host 'github.com' is known and matches the RSA host key.
debug1: Found key in /root/.ssh/known_hosts:1
Warning: Permanently added the RSA host key for IP address '192.30.252.129' to the list of known hosts.
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /root/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 279
debug1: key_parse_private_pem: PEM_read_PrivateKey failed
debug1: read PEM private key done: type <unknown>
debug1: read_passphrase: can't open /dev/tty: No such device or address
debug1: Trying private key: /root/.ssh/id_dsa
debug1: Trying private key: /root/.ssh/id_ecdsa
debug1: No more authentication methods to try.
Permission denied (publickey).

I have tried chmod 600 and chmod 700 on the priv/public keys, this did not help.

Any clues? Has anyone succeeding in running go get that fetches from private repos on debian from docker?

orcaman
  • 6,263
  • 8
  • 54
  • 69
  • Are you running your `go get` commands as root? You should use `go get` with your user account, and store your key in `/home/yourAccount`. – VonC Oct 02 '14 at 15:08
  • Would checking the permissions and the shared preferences as in http://stackoverflow.com/a/19798820/6309 help? – VonC Oct 03 '14 at 06:03

5 Answers5

15

I figured this out after a bit of hacking around. Not an ideal solution as it involves installing SSH, plus building a private key into the container. This example is based on the official Docker golang image (Debian Wheezy):

The main difference to your example is that you need a git config command to force ssh instead of the default https.

FROM golang

RUN apt-get update && apt-get install -y ca-certificates git-core ssh

ADD keys/my_key_rsa /root/.ssh/id_rsa
RUN chmod 700 /root/.ssh/id_rsa
RUN echo "Host github.com\n\tStrictHostKeyChecking no\n" >> /root/.ssh/config
RUN git config --global url.ssh://git@github.com/.insteadOf https://github.com/

ADD . /go/src/github.com/myaccount/myprivaterepo

RUN go get github.com/myaccount/myprivaterepo
RUN go install github.com/myaccount/myprivaterepo
t j
  • 7,026
  • 12
  • 46
  • 66
  • 10
    The problem that I have with this solution is that it requires my private ssh key to be copied into the copied into the directory where my source code is located. Is there a way to avoid this? – nojo May 02 '15 at 21:24
  • Agree with nojo, the private key shouldn't be let there inside the container, after all, it is "private" – Ualter Jr. Nov 24 '20 at 16:03
  • 1
    Perhaps using multi-staged builds would solve this as the credentials are only kept in the build stage and not the final image? See https://docs.docker.com/develop/develop-images/multistage-build/ for more details – matt Dec 06 '21 at 14:17
13

go get is trying to use https, completely ignoring ssh.

You will have to setup ~/.netrc:

ADD priv/.netrc /root/.netrc

Where netrc looks like:

machine github.com login github-username password github-password

ref:

Community
  • 1
  • 1
OneOfOne
  • 95,033
  • 20
  • 184
  • 185
  • thanks for the help. thing is - on my machine (Mac) go get works perfectly fine with the private repo (authentication proceeds smoothly). the problem only occurs on the debian docker image that I'm working on. also, please notice that the errors are observed before go get even happens (ssh-keyscan output shows there are errors as I have posted). – orcaman Oct 02 '14 at 16:13
  • @orcaman `read PEM private key done: type ` might be a corrupt private key. – OneOfOne Oct 02 '14 at 16:45
  • 1
    "go get is trying to use https, completely ignoring ssh." Actually that's fixable with `RUN git config --global url."git@github.com:".insteadOf "https://github.com"` – Eksapsy May 17 '21 at 09:46
7

In the newest version of golang (v1.11) there are now modules.

To quote the source:

A module is a collection of related Go packages that are versioned together as a single unit. Most often, a single version-control repository corresponds exactly to a single module.

Using the latest version of golang will allow you to have dependencies that are in private repositories. Essentially by running the $ go mod vendor command will create a vendor directory locally for all external dependencies. Now making sure your docker image has Golang v1.11, you will update your Dockerfile with the following:

WORKDIR /<your repository>

COPY . ./
JohnAllen
  • 7,317
  • 9
  • 41
  • 65
danielsmith1789
  • 1,056
  • 1
  • 12
  • 25
6

Elaborating on OneOfOne's ~/.netrc answer, this is what I am doing with Jenkins on linux:

FROM golang:1.6

ARG GITHUB_USER=$GITHUB_USER
ARG GITHUB_PASS=$GITHUB_PASS

# Copy local package files to the container's workspace.
ADD . /go/src/github.com/my-org/my-project
WORKDIR /go/src/github.com/my-org/my-project/

# Build application inside the container.
RUN echo "machine github.com\n\tlogin $GITHUB_USER\n\tpassword $GITHUB_PASS" >> ~/.netrc && \
    go get github.com/tools/godep && \
    go get github.com/onsi/ginkgo/ginkgo && \
    godep restore && \
    ginkgo -r --randomizeAllSpecs --randomizeSuites --failOnPending && \
    godep go install && \
    rm -f ~/.netrc

ENTRYPOINT /go/bin/my-project

EXPOSE 8080

The docker build command is:

docker build \
    --build-arg GITHUB_USER=xxxxx \
    --build-arg GITHUB_PASS=yyyyy \
    -t my-project .

The two ARG directives map --build-args so docker can use them inside the Dockerfile.

The first and last lines of RUN create and remove the ~/.netrc.

In Jenkins, I use the same creds from git pull in the build command.

In this strategy, the password is not echoed during the docker build process and not saved on any layer of your docker image. Also note that the gingko test results are printed to console during the build.

Alexis Wilke
  • 19,179
  • 10
  • 84
  • 156
Steve Tarver
  • 3,030
  • 2
  • 25
  • 33
  • 2
    The problems with this strategy is 1. History records your username and password in plain text. So anyone with any kind of access to your computer can just run "history" and look at your credentials, accidentally or not. 2. You cant use that strategy in docker-compose. If you do, you expose your credentials in a file now in plain text, and even worse, with the risk of accidentally committing the change in the file. It's a really deal breaker if you want docker-compose – Eksapsy May 17 '21 at 09:48
  • @Eksapsy - decent security concerns - each user should adjust to their risk tolerance . Docker & GitHub has advanced quite a bit in 5 years and provide several secure alternatives but we don't know enough in this questions to suggest - only to get r' goin. – Steve Tarver May 18 '21 at 20:33
  • 1
    The issue is annoying in general. There doesn't seem to be a good generic solution for all and its frustrating. Docker should definitely advance on this issue a little bit more and provide us a good solution. In any case, good job on trying to provide a solution, and it may work for some of us. I just personally find it too risky and security concerning for my taste, especially with docker-compose that most of us use nowadays. – Eksapsy May 20 '21 at 08:24
  • i think it's not a good solution for fixing this problem, you can only configure your git for using `personal access token` – Amirreza Saki Feb 06 '22 at 10:27
5

i had this problem in GitHub and fixed it using a personal access token:

  1. Use ARG for your Dockerfile vars (inputs)
  2. Configure your git with GitHub personal access token

GITHUB_PAT is the GitHub personal access token:

FROM golang:1.17 as builder

ARG GITHUB_PAT

WORKDIR /your-app
COPY go.mod .
COPY go.sum .
RUN git config --global url."https://${GITHUB_PAT}:x-oauth-basic@github.com/".insteadOf "https://github.com/"
RUN go mod download
COPY . .
RUN go build -ldflags '-w -s' -o ./out ./main.go


FROM golang:1.17

WORKDIR /app

COPY --from=builder /your-app/out ./
WORKDIR /app/

ENTRYPOINT [ "./out" ]
Benjamin W.
  • 46,058
  • 19
  • 106
  • 116
Amirreza Saki
  • 601
  • 7
  • 8