How to run or deploy an app with a git dependency to a private repo

Question

My application has a Git dependency to a private BitBucket repository.

  my_package:
    git:
      url: git@bitbucket.org:myuser/mypackage.git

When I run

gcloud --verbosity debug preview app run app.yaml

I get

Resolving dependencies...
Git error. Command: git clone --mirror git@bitbucket.org:myuser/my_package.git /root/.pub-cache/git/cache/my_package-6fe77906161a7a9252973e29e39a4149b75f9a7e
error: cannot run ssh: No such file or directory
fatal: unable to fork

I guess adding an ADD instruction to the Dockerfile would be a viable workaround.
The repo needs to be checked out to a local directory to make this work of course.

I tried:

FROM google/dart-runtime
ADD ../my_package ../my_package

https://docs.docker.com/reference/builder/#add says

The <src> path must be inside the context of the build; you cannot ADD 
../something /something, because the first step of a docker build is to
send the context directory (and subdirectories) to the docker daemon.

It seems I have to move the ..my_package directory into the my_app directory. This isn't beautiful :-(

When I add a bogus line to the Dockerfile run fails but if I add an ADD ... instruction it seems to be ignored entirely.

error: cannot run ssh: No such file or directory Looks like there is just some problem with ssh? — Robert, Nov 12 '14 at 10:37
Is this a problem with private repos only or any repo that uses SSH? — Pixel Elephant, Nov 12 '14 at 19:08
I think it is only with private repos. Currently I try to create a Docker image from scratch with installed ssh-client and a SSH deployment key, but I'm not there yet. It seems pretty difficult to get files into a Docker image which are not in the directory containing the `Dockerfile`. — Günter Zöchbauer, Nov 12 '14 at 19:10
Can you not use a different URL, that uses the http protocol? Otherwise I think you'll have to instruct Docker to install ssh and maybe provide a ssh key like you're trying to do. — James Hurford, Nov 12 '14 at 23:13
I think I got it working. I'll verify it tomorrow and add an answer. — Günter Zöchbauer, Nov 12 '14 at 23:15
Günter Zöchbauer Can you please confirm if there is a different solution for this problem now? — vzurd, Apr 09 '21 at 21:43

score 4 · Accepted Answer · edited Mar 20 '17 at 10:18

Update 2

My previous solution is still very inconvenient because I have to check in every time before I re-launch the app.

With great support I found a much more convenient solution. Instead of an symlink I mount the directory. See https://superuser.com/questions/842642 for more details. I don't know if and how this can work on other oSes (Win, OX X, ...)

I mount ../my_package to docker/my_package (instead of a symlink) and use this Dockerfile:

FROM google/dart

WORKDIR /app
ENV DART_SDK /usr/lib/dart

ADD dart_run.sh /dart_runtime/

RUN chmod 755 /dart_runtime/dart_run.sh && \
 chown root:root /dart_runtime/dart_run.sh

ADD pubspec.yaml /app/
ADD pubspec.lock /app/
ADD docker/my_package /my_package
RUN pub get
ADD . /app/
RUN pub get --offline

## Expose ports for debugger (5858), application traffic (8080)
## and the observatory (8181)
EXPOSE 8080 8181 5858

CMD []
ENTRYPOINT ["/dart_runtime/dart_run.sh"]

Update 1

It turns out just serving the .git directory of the checked out package though git-daemon is a much more convenient solution.
All I had to do was setting all up according to the docs at https://www.dartlang.org/cloud/ and use a git dependency in pubspec.yaml to this repo served by git-daemon.

  my_package:
    git:
      url: git://192.168.2.96/my_package
      ref: test

This url works when working locally and also within the Docker container.

Original

I can run my app with this Dockerfile

FROM google/dart

WORKDIR /app

RUN \
  apt-get update && \
  apt-get install -y openssh-client

ADD tool/bitbucket_deployment_key /root/.ssh/id_rsa

RUN \
  mkdir -p /root/.ssh && \
  echo "Host bitbucket.org" >> /root/.ssh/config && \
  echo "    StrictHostKeyChecking no" >> /root/.ssh/config && \
#  ssh-keyscan -t rsa -H bitbucket.org,131.103.20.167 >> /root/.ssh/known_hosts && \
  chmod 400 /root/.ssh/id_rsa && \
  eval $(ssh-agent) && \
  ssh-add /root/.ssh/id_rsa

RUN \
  git clone git@bitbucket.org:myuser/my_package.git /my_package --branch test && \
  rm /root/.ssh/id_rsa

#ENV DART_SDK /usr/lib/dart

ADD dart_run.sh /dart_runtime/

RUN chmod 755 /dart_runtime/dart_run.sh && \
  chown root:root /dart_runtime/dart_run.sh

ADD pubspec.yaml /app/
ADD pubspec.lock /app/
RUN pub get
ADD . /app/
RUN pub get --offline

# Expose ports for debugger (5858), application traffic (8080)
# and the observatory (8181)
EXPOSE 8080 8181 5858

CMD []
ENTRYPOINT ["/dart_runtime/dart_run.sh"]

I created id_rsa with ssh-keygen without a passphrase. This is the reason I delete the file from the image after the git clone command. It isn't used afterwards anyway.

In my BitBucket repo I added id_rsa.pub as a deployment key.

clever idea. Did you try simply making `my_package` a git dependency? Since you installed the ssh key the pub get should resolve it then right? Unless of course, your git repo contains more than one dart package (as mine does) — Anders, Nov 15 '14 at 23:36
As far as I know a git dependency with a repo containing more than one package is generally not supported by `pub`. I got it working with a git dependency to the bitbucket repo. The problem was the authentication but I got it working (see my answer). But I figured that it is more convenient to make the container load the files from the checked out directory because I don't have to check in every change to make it take effect in the container. — Günter Zöchbauer, Nov 16 '14 at 08:55
Yes I've just cut over to using your git clone trick above and it is tedious having to check in all the time. Also it runs into docker image cloning so that breaks things if you push new changes to your repo. I'll need to look at restructuring my code so I can go back to working with local projects. I might try symlinks — Anders, Nov 16 '14 at 11:39
Actually I have to check in locally to make it work, but I don't need to push. Basically any method copying over the network could be used instead. — Günter Zöchbauer, Nov 16 '14 at 13:13
I'm going back to just normal paths but with the docker file up a couple of directories. Much easier to work that way but not too pretty — Anders, Nov 16 '14 at 22:30
This way you need to change the `run.sh` file used to invoke `server.dart`, don't you? — Günter Zöchbauer, Nov 17 '14 at 05:02
Here is a similar question (not Dart related) http://stackoverflow.com/questions/26639968 — Günter Zöchbauer, Nov 17 '14 at 05:18
Yes and app.yaml etc so not pretty. I'm considering automating those changes with grinder at some point so as it copies those files up to the parent directory it modifies the paths appropriately — Anders, Nov 17 '14 at 05:44
@GünterZöchbauer Can you please confirm if there a better solution for this now ? — vzurd, Apr 02 '21 at 17:44

Cristian Almstrand · Answer 2 · 2014-11-21T01:00:59.893

0

Apologies for adding this as an answer but am unable to comment without more reputation:

Please vote on google-cloud-sdk issue 93 if you would like this to be fixed:
https://code.google.com/p/google-cloud-sdk/issues/detail?id=93

edited Nov 21 '14 at 01:00

answered Nov 20 '14 at 22:06

Cristian Almstrand

683
5
16

How to run or deploy an app with a git dependency to a private repo

2 Answers2

Update 2

Update 1

Original