15

I am trying to clone a repo that has submodules in it. The main repo is cloning fine but when I do git submodule update --init --recursive in the dockerfile the submodules throws and error.

fatal: clone of 'git@github.com:jkeys089/lua-resty-hmac.git' into submodule path '/tmp/third-party/lua-resty-hmac' failed
Failed to clone 'third-party/lua-resty-hmac'. Retry scheduled
Cloning into '/tmp/third-party/lua-resty-jwt'...
load pubkey "/root/.ssh/id_rsa": invalid format
Warning: Permanently added the RSA host key for IP address '140.82.118.3' to the list of known hosts.
Load key "/root/.ssh/id_rsa": invalid format
git@github.com: Permission denied (publickey).

In the image I have this

# authorise ssh host
RUN mkdir /root/.ssh/ \
    && chmod 700 /root/.ssh \
    && ssh-keyscan github.com > /root/.ssh/known_hosts

# add key and set permission
RUN echo "${SSH_PRIVATE_KEY}" >> /root/.ssh/id_rsa \
    && echo "$ssh_pub_key" > /root/.ssh/id_rsa.pub \
    && chmod 600 /root/.ssh/id_rsa.pub \
    && chmod 600 /root/.ssh/id_rsa

I have no control of the submodules. I am not sure if I can change from git@github.comto https to get submodules.

I even tried using the GITHUB_TOKEN route 

# start up git and clone
RUN git config --global url."https://${GITHUB_TOKEN}@github.com/".insteadOf "https://github.com/" \
    && git clone https://github.com/GluuFederation/gluu-gateway.git /tmp \
    && cd /tmp/ \
    && git submodule update --init --recursive

And below is the part of the build command. build --build-arg GITHUB_TOKEN=${GITHUB_TOKEN} --build-arg SSH_PRIVATE_KEY="$(cat ~/.ssh/id_rsa)" --build-arg ssh_pub_key="$(cat ~/.ssh/id_rsa.pub)"

Please help out on this. It's very frustrating. :(

Shammir
  • 927
  • 4
  • 17
  • 32
  • The file you're passing as `SSH_PRIVATE_KEY` is the public key. Note that doing this will compromise your key pair, since anyone who gets the image can trivially extract it, and there are several other disadvantages of running `git` commands inside the Dockerfile; I'd set this up (clone the repository, check out the specific branch I want to build, ...) on the host before running `docker build`. – David Maze Dec 28 '19 at 10:33
  • You can use the same `insteadOf` trick to change `git@github.com:...` into an https URL, but see David Maze's comment. – torek Dec 28 '19 at 10:40
  • My thinking was that if this works, I can use multi-stage build to avoid leaving traces of private keys in build history. – Shammir Dec 28 '19 at 10:48
  • I have updated the question - in regards to pub key in build command. I am still getting the same error. – Shammir Dec 28 '19 at 10:51
  • What is the text in the first line of your private key (the part between the dashes), and what OS are you running in the container? – bk2204 Dec 28 '19 at 20:56

8 Answers8

14

Another possible gotcha is if you're using a Makefile to run the docker build command. In that case the command in the Makefile would look something like:

docker-build:
    docker build --build-arg SSH_PRIVATE_KEY="$(shell cat ~/.ssh/id_rsa)"

Make unfortunately replaces newlines with spaces (make shell)

This means that the ssh key which is written into the container has a different format, yielding the error above.

I was unable to find a way to retain the newlines in the Makefile command, so I resorted to a workaround of copying the .ssh directory into the docker build context, copying the files through the Dockerfile, then removing them afterwards.

Richard Fisher
  • 141
  • 1
  • 2
11

If the key is "invalid format", try and regenerate it with the old PEM format.

ssh-keygen -m PEM -t rsa -P ""

Make sure to add the public key to your GitHub account for proper authentication.

The OP Shammir adds in the comments:

I think the issue is that nothing is being copied from host machine to docker image during build.

In "docker build --build-arg SSH_PRIVATE_KEY="$(cat ~/.ssh/id_rsa)" returning empty", Shammir uses dockito/vault to manage the private key, but also configure it to "AddKeysToAgent": that is not needed if the private key is not passphrase protected (as in my command above)

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • I think the issue is that nothing is being copied from host machine to docker image during build. I have asked that question here https://stackoverflow.com/questions/59541238/docker-build-build-arg-ssh-private-key-cat-ssh-id-rsa-returning-empty?noredirect=1#comment105252129_59541238 – Shammir Dec 31 '19 at 08:38
  • 1
    @Shammir I have included your comment and solution in the answer for more visibility. – VonC Dec 31 '19 at 10:00
10

For reference only.

I created a private key file key.id_rsa manaully and pasted content to it. But when I used it to clone git repository:

$ git clone my_repo
Cloning into 'my_repo'...
Load key "/path/to/key.id_rsa": invalid format

The key's content is definitely correct. So then I tried compare my key with the other generated key by ssh-keygen, it's truely invalid format.

Just no end of new line in my key.

After added new line to the end of key, all worked delightful~ What a suprise!

[Update]: Remember use \n instead of \r\n even on windows.

LitileXueZha
  • 512
  • 6
  • 11
  • I guess this is just a workaround for a problem that would not come up using `COPY` instead of `echo`? Then you do not need to manually add `\n`. See [my answer](https://stackoverflow.com/a/66648403/11154841) of the same thread. – questionto42 Aug 12 '21 at 10:43
6

This answer is for Windows users (haven't tried this on linux).

I searched many answers and articles but still was getting invalid format error while building my docker image.

The actual reason is that when we pass the content of the private key file as an argument, they are passed in a single line. The escape characters are converted to space which is basically invalid format for the key. To avoid this error, there are two ways of passing private key to the docker image:

  1. Use COPY command in the docker file to copy the private key file and use it in the docker image. This is not considered as a good option as it may expose your private key. Example: COPY id_rsa /root/.ssh/id_rsa
  2. This is a kind of hack which I used and it worked. Open the private key file in a text editor and add \n at the end of every line in the private key and join every line to create the whole key in one line. For example, your generated key looks like this:

-----BEGIN OPENSSH PRIVATE KEY----- b3BlbnNzaC1redjEAAAAABG5vvmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW QyNTUxOQAAACdmF7/Vo4m2FWPf+8uZRRF88dnsyj+z+lCWNWBrT8gAAAJh1tssodbbL -----END OPENSSH PRIVATE KEY-----

make it look like this:

-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1redjEAAAAABG5vvmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW\nQyNTUxOQAAACdmF7/Vo4m2FWPf+8uZRRF88dnsyj+z+lCWNWBrT8gAAAJh1tssodbbL\n-----END OPENSSH PRIVATE KEY-----\n

Save it in the key file and pass the modified file in the arguments while building docker image.

Raghav Mundhra
  • 168
  • 1
  • 4
  • This is a repetition of [my answer](https://stackoverflow.com/a/66648403/11154841) of the same thread. – questionto42 Aug 16 '21 at 11:40
  • Thanks! After 2 hours of fighting, adding the \n to the private ssh key on windows fixed the problem. The question is still, is there maybe a way to generate the key in the correct format so it works on all systems? Since a long time this was the first time I experienced this issue. – Daniel Kemeny Sep 08 '21 at 05:59
  • @DanielKemeny, actually the key is generated in correct format. There is nothing wrong with it. The formatting of the key gets changed when we pass it through --build-arg and use it through "echo" in dockerfile. The argument is passed in a single line, hence corrupting the key. – Raghav Mundhra Sep 14 '21 at 04:48
4

Make sure there is a \n after the last line. Took me too long to figure this out.

Leon
  • 181
  • 1
  • 3
2

Do not use echo "${SSH_PRIVATE_KEY}" >> /root/.ssh/id_rsa to pass the private key (same for the public key). I had a similar error Load key "/root/.ssh/id_rsa": invalid format when I tried

RUN echo "$ssh_prv_key" > /root/.ssh/id_rsa &&     chmod 600 /root/.ssh/id_rsa

This led to errors like identity file /root/.ssh/id_rsa type -1 invalid format and read_passphrase: can't open /dev/tty.

The right way would be to use

COPY id_rsa /root/.ssh/id_rsa
RUN chmod 600 /root/.ssh/id_rsa

The solution explained: my private key was wrongly formatted - instead of many lines, it was passed as a one-liner, and you might have any other format issue like a forgotten "-" at the start or end, or something wrong at the end of the lines, like a missing newline format or an additional letter at the end of a line.

See Dockerfile: clone repo with passwordless private key. Errors: “authentication agent” or “read_passphrase: can't open /dev/tty” for more details, with the main idea from Add private key to ssh-agent in docker file, which again had the idea from Gitlab CI/Docker: ssh-add keeps asking for passphrase.

questionto42
  • 7,175
  • 4
  • 57
  • 90
  • 1
    For this, you will have to keep the Secret key file in the same directory everytime you build your image which can be a security concern (think if you by mistake push the key to GitHub or any version control) – Chayan Bansal Aug 18 '21 at 04:57
  • 1
    @ChayanBansal You can also copy the key from a folder that is not in the github directory. Beyond that, you must delete the private key after each use since it is saved in plain text in the image. This is awkward, yet, the main thing here is not to be efficient, but to make it run at all in a Dockerfile. There is also a way to avoid saving the private key in the image by setting up a second image on top of the first and copy only what is needed, see [Using SSH keys inside docker container](https://stackoverflow.com/a/66648529/11154841). Recently, you would rather use deployment keys perhaps. – questionto42 Aug 18 '21 at 07:49
  • Invalid solution for CI/CD pipelines. We can't store the id_rsa in the repository. CloudBuild for instance cannot be used in this scenario. – Nathan McKaskle May 09 '23 at 19:07
2

In most cases, you should not add any ssh key in your container due to security risks since any person can get the ssh key if they have access to your image.

Similarly, using ARG is not recommended by Docker either, as you can retrieve the secrets using docker history <image>. The secret is embedded in the layer on the build. You can find more details here https://docs.docker.com/engine/reference/builder/#arg

The recommended approach is to use CI/CD to clone the GitHub repo to the local folder, then use COPY operation to copy the code into your image. But if you absolutely want to clone your GitHub repo, there are two approaches:

  1. Mount secret into a volume during build https://render.com/docs/docker-secrets. Please note that the secret mount has root permission, therefore you will get a permission error if you read the secrets as non-root users.
  2. Using multi-stage build. You can have gitsync image in the first stage, clone the repo, then copy the content of the repo to your image in the second stage. https://docs.docker.com/build/building/multi-stage/. This way you can load secret into the first stage, clone the repo, then copy the artifact to the later stage. The final stage image will not have any trace of your secrets if you do it correctly.

An example of cloning a private library and install it in the final image:

# Clone python library from a private repo, then copy to the python image and install as library
# docker build -t <img_tag> --build-arg BOT_GITHUB_USER=$BOT_GITHUB_USER --build-arg BOT_GITHUB_PRIVATE_TOKEN=$BOT_GITHUB_PRIVATE_TOKEN  .
# multi stage build, clone lib repo then copy to main image
FROM k8s.gcr.io/git-sync/git-sync:v3.6.1 as gitsync
ARG BOT_GITHUB_USER
ARG BOT_GITHUB_PRIVATE_TOKEN

ENV GIT_SYNC_USERNAME=$BOT_GITHUB_USER
ENV GIT_SYNC_PASSWORD=$BOT_GITHUB_PRIVATE_TOKEN
ENV GIT_SYNC_BRANCH=dev
RUN /git-sync --repo=https://github.com/yourrepo/mylib --one-time


FROM python

RUN mkdir mylib

COPY --from=gitsync /tmp/git/mylib mylib/

RUN pip install -e mylib/
Ohhimark
  • 21
  • 3
-1

In my case it was because of missing trailing empty line.

MingalevME
  • 1,827
  • 1
  • 22
  • 19