Is there a way to automatically deploy to GCE based on a new image being created in Google Container Registry?

Question

I have a Kubernetes deployment on GCE, which I'd like to get automatically updated based on new images being created in Google Container Registry (ideally via a Build Trigger). Is there a way to do that?

Thanks in advance.

-Mark

Answer 1

I was able to do this using GCR and Cloud Builder with a cloudbuild.yaml file like the below. For it to work, the service account with a name like xyz@cloudbuild.gserviceaccount.com had to have the IAM permissions assigned by clicking Project -> Editor. This is required so that the Cloud Build service can make SSH keys and add them to your GCE metadata to allow Cloud Builder to SSH in. This SSHing is the big work-around to effectively run any command on your GCE VM server.

steps:
# Build Docker image: docker build -f Dockerfile -t gcr.io/my-project/my-image:latest .
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-f', 'Dockerfile', '-t', 'gcr.io/my-project/my-image:latest', '.']

# Push to GCR: gcloud docker -- push gcr.io/my-project/my-image:latest
- name: 'gcr.io/cloud-builders/docker'
  args: ['push', 'gcr.io/my-project/my-image:latest']

# Connect to GCE server and pull new image
- name: 'gcr.io/cloud-builders/gcloud'
  args: ['compute', 'ssh', '$_SERVER', '--zone', '$_ZONE', '--command', 'gcloud docker -- pull gcr.io/my-project/my-image:latest']

# Connect to server and stop current container
- name: 'gcr.io/cloud-builders/gcloud'
  args: ['compute', 'ssh', '$_SERVER', '--zone', '$_ZONE',  '--command', 'docker stop my-image']

# Connect to server and stop current container
- name: 'gcr.io/cloud-builders/gcloud'
  args: ['compute', 'ssh', '$_SERVER', '--zone', '$_ZONE',  '--command', 'docker rm my-image']

  # Connect to server and start new container
- name: 'gcr.io/cloud-builders/gcloud'
  args: ['compute', 'ssh', '$_SERVER', '--zone', '$_ZONE',  '--command', 'docker run  --restart always --name my-image -d -p 443:443  --log-driver=gcplogs  gcr.io/my-project/my-image:latest']


substitutions:
  _SERVER: 'my-gce-vm-server'
  _ZONE: 'us-east1-c'

Bonus Pro Tips:

1. the substitutions are nice in case you prop up a new server some day and want to use it instead
2. using --log-driver=gcplogs makes your Docker logs show up in your Google Cloud Console's Stackdriver Logging in the appropriate "GCE VM Instance". Just be sure to have "All logs" and "Any Log Level" selected since Docker logs have no log level and are not syslog or activity_log messages

Answer 2

Another option: some of our users use the kubectl build step to trigger a deployment at the end of their build.

You can call any kubectl command in your build step, provided that you have set up the proper IAM permissions to do so as part of a build. (See the README.) This example calls kubectl get pods.

Note that images are only automatically pushed at the end of a completed build, so to build an image and deploy it in one build, you'll need to insert your own docker push build step prior to your deployment step.

Answer 3

You can use Google Cloud pub/sub to listen to changes in Google Container Registry. This page gives an overview of this feature. You may want to use the push model for your application.

However, please note that this is an alpha feature and its behavior may change in future releases.

If you don't want the external control provided by pub/sub, your build script should do the following,

1. Tag the image and upload to container registry
2. Update image version in deployment scripts
3. Run the deployment script which in turn will pull the latest image