2

I'm deploying to app engine using the custom runtime and Im noticing that when I deploy, app engine will fully re-build my Dockerfile without any caching. This causes deployments to take much longer. I'm not changing my Dockerfile between deploys. Only my application code is changing. Here is my Dockerfile:

FROM ubuntu
EXPOSE 8080

RUN apt-get update
RUN apt-get install -yq python-crypto python-openssl libffi-dev libssl-dev
RUN pip install --upgrade pip
RUN pip install gunicorn==19.4.5
RUN pip install Flask==0.10.1
RUN pip install PyMySQL==0.7.2
RUN pip install alembic==0.8.5
RUN pip install Flask-Migrate==1.8.0
RUN pip install Flask-CORS==2.1.2
RUN pip install PyCrypto==2.6.1
RUN pip install requests==2.9.1
RUN pip install --upgrade cffi
RUN pip install google-api-python-client==1.5.0
RUN pip install gcloud==0.11.0

# Ensure that Python outputs everything that's printed inside
# the application rather than buffering it.
ENV PYTHONUNBUFFERED 1

ADD . /app/
WORKDIR /app

ENTRYPOINT ["gunicorn", "-b", ":8080", "server:app"]

Is there a way to speed up my deploys?

sthomps
  • 4,480
  • 7
  • 35
  • 54
  • Are you using [Container Builder API](https://cloud.google.com/container-builder/docs/)? GCE-based remote builds? Local Docker builds? – asamarin Apr 16 '16 at 23:04
  • Im using app engine managed vm (which I think uses container builder). Im deploying using `gcloud --project myproject preview app deploy --version v1 app.yaml` – sthomps Apr 16 '16 at 23:08

2 Answers2

1

The goal of the deployment is building the docker image to be executed as your app.

The dockerfile only contains the docker image build instructions, so the fact that its content doesn't change means nothing - the build instructions still need to be executed to obtain the new docker image.

The only way to speedup the docker image build would be to store the unchanged temporary result (say current FROM image plus all the pip packages installed) as an interim docker image to be referenced as the FROM for another dockerfile which would only update your app's code. But AFAIK GAE doesn't (at least not yet) allow saving such custom intermediate images for re-use as base images in building other custom images.

Dan Cornilescu
  • 39,470
  • 12
  • 57
  • 97
  • The troubleshooting page suggests that there should be some caching though? (https://cloud.google.com/appengine/docs/flexible/custom-runtimes/troubleshoot) – sthomps Apr 17 '16 at 06:04
  • 1
    Ah, that cache exists, but it's a shared cache which you can't control. You *might* get its benefits or not, depending on what you're building and how often you do so. The build log should indicate if it benefited from the cache (the `Using cache` messages) or not, as seen in this post: http://stackoverflow.com/questions/34500213/how-can-i-speed-up-rails-docker-deployments-on-google-cloud-platform. – Dan Cornilescu Apr 17 '16 at 12:55
  • It *might* be possible to control the cache, see my updated answer to that question. – Dan Cornilescu Apr 17 '16 at 21:48
1

If you have a local docker environment that is configured correctly you could use the --docker-build local flag to perform a local build.

--docker-build DOCKER_BUILD

Perform a hosted ('remote') or local ('local') Docker build. To perform a local build, you must have your local docker environment configured correctly. The default is a hosted build.

-- https://cloud.google.com/sdk/gcloud/reference/preview/app/deploy#--docker-build

Jeffrey Godwyll
  • 3,787
  • 3
  • 26
  • 37
  • Im not sure this did anything. In the console log I can still see `REMOTE BUILD OUTPUT` and its going through all the steps from the Dockerfile – sthomps Apr 17 '16 at 03:24