How do I customise a Google App Engine Managed VM with a Standard Runtime?

Question

I would like to customise a (Python) Standard Runtime Managed VM. In theory, this should be possible by adding some extra commands to the VM Dockerfile.

Google's documentation states that a VM Dockerfile is automatically generated when the App is first deployed;

If you are using a standard runtime, the SDK will create a Dockerfile for you the first time you run the gcloud preview app deploy commands. The file will exist in a predetermined location:

• If you are developing in Java, the Dockerfile appears in the root of the compiled Web Application Archive directory (WAR)
• If you are developing in Python or Go, the Dockerfile appears in the root of your application directory.

And that extra commands can indeed be added;

You can add more docker commands to this file, while continuing to run and deploy your app with the standard runtime declaration.

However in practice the Dockerfile is automatically deleted immediately after deployment competes, preventing any customisation.

Has anyone managed to add Dockerfile commands to a Managed VM with a Standard Runtime? Any help would be gratefully appreciated.

Answer 1

I tried the same thing and did not succeed. There is however an equivalent way of doing this that I fell back to.

You can create a custom runtime that mimics the standard runtime.

You can do this because Google provides the Docker base images for all the standard runtimes. Mimicking a standard runtime is therefore simply a matter of selecting the right base image in the Dockerfile of the custom runtime. For the standard Python App Engine VM the Dockerfile is:

FROM gcr.io/google_appengine/python-compat
ADD . /app

Now that you have recreated the standard runtime as a custom runtime, you can modify the Dockerfile to make any customizations you need.

Important Note

The development server does not support custom Dockerfiles (you will get an error about --custom-entrypoint), so you have to move your test environment to App Engine servers if you are doing this. I think this is true regardless of whether you are using a standard runtime and customizing the Dockerfile or using a custom runtime. See this answer.

Answer 2

A note about the development server not working with custom runtimes - dev_appserver.py doesn't deal with Docker or Dockerfiles, which is why it complains about needing you to specify --custom_entrypoint. However as a workaround you can manually set up the dependencies locally. Here's an example using 'appengine-vm-fortunespeak' which uses a custom runtime based on python-compat:

$ git clone https://github.com/GoogleCloudPlatform/appengine-vm-fortunespeak-python.git 
$ cd appengine-vm-fortunespeak-python 

# Local dependencies from Dockerfile must be installed manually 
$ sudo pip install -r requirements.txt 
$ sudo apt-get update && install -y fortunes libespeak-dev 

# We also need gunicorn since its used by python-compat to serve the app 
$ sudo apt-get install gunicorn 

# This is straight from dev_appserver.py --help 
$ dev_appserver.py app.yaml --custom_entrypoint="gunicorn -b localhost:{port} main:app" 

Note that if you are using any of the non -compat images, you can run your app directly using Docker since they don't need to emulate the legacy App Engine API, for example using 'getting-started-python' which uses the python runtime:

$ git clone https://github.com/GoogleCloudPlatform/getting-started-python.git 
$ cd 6-pubsub 

# (Configure the app according to the tutorial ...)

$ docker build . 
$ docker images # (note the IMAGE_ID) 
$ docker run -p 127.0.0.1:8080:8080 -t IMAGE_ID

Try the above with any -compat images and you will have problems - for example on python-compat you'll see initialization errors in runtime/google/appengine/tools/vmboot.py. It needs to be run on a real Managed VM instance.