How to automatically delete old Google App Engine version instances?

Question

I'm experimenting with more cost effective ways to deploy my Rails apps, and went through the Ruby Starter Projects to get a feel for Google Cloud Platform.

It's almost perfect, and certainly competitive on price, but I can't figure out how to automatically delete old deployed version instances after redeploying.

ie: let's say I have one version of each instance running:

And then I make a change to my app and redeploy with:

$ gcloud preview app deploy app.yaml worker.yaml --promote

So now I have two versions of each instance deployed (as Google switches between them intelligently, I'd assume):

But now what? Will these instances ever turn off by themselves? The best way I've found of getting rid of them so far is from the tutorial's Deleting the project page:

Deleting app versions

You can find the list of your app versions in the Versions page. To delete the non-default versions, select the check boxes and then click Delete.

Is everyone on Google App Engine just manually deleting the old deployments of their apps?

Answer 1

We deploy an App Engine instance from Cloud Build, which is a common deployment pattern. At the end of the cloudbuild.yaml, we specify the following commands as the last build step, which deletes every version except the last 5 versions:

versions=$(gcloud app versions list \
  --service default \
  --sort-by '~VERSION.ID' \
  --format 'value(VERSION.ID)' | sed 1,5d)
for version in $versions; do
  gcloud app versions delete "$version" \
    --service default \
    --quiet
done

Answer 2

Just to throw my solution here after reading all the others provided:

gcloud app versions list --format="value(version.id)" --sort-by="~version.createTime" | tail -n +6 | xargs -r gcloud app versions delete --quiet

Main benefit is that it can still run successfully even if theres 5 or less existing versions.

Answer 3

To stop all instances of all non-default versions of all modules (independently of what languages those modules are in), you could add a small control module, written in Python, using the modules API:

from google.appengine.api.modules import modules

# core logic (inside a cron or other handler)
for m in modules.get_modules():
    dv = modules.get_default_version(m)
    for v in modules.get_versions(m):
        if v != dv: modules.stop_version(m, v)

This doesn't delete the non-default versions (the modules API does not appear to currently support deletion), but does ensure that none of their instances are running (so no charges would be incurred for them).

This core logic is meant for you to wrap it inside a handler, so you can trigger it as required, for example in a cron job, or in a "regular" handler that you trigger from the outside (with appropriate auth) e.g. via wget or curl within your bash scripts.

I don't believe there's a Ruby version of Python's google.appengine.api.modules.modules , but, I could be wrong... I just couldn't find one. However, a simple Python-coded module should let you control modules coded in whatever other App Engine language (since App Engine lets you mix and match modules coded in different languages).

Answer 4

The following removes all version except 5 most recent: Uses ghead (on macOS, brew install coreutils), replace with head for linux.

gcloud app versions delete `gcloud app versions list | sed 's/  */:/g' | cut -f 2 -d : | tail -n +2 | ghead -n -5 | tr "\n" " "`

Answer 5

For those not familiar with xargs syntax, I took the very nice answer from @bkanuka and split it in two lines. It keeps last 5 versions and works even if there is less than 5 versions deployed:

    VERSIONS=$(gcloud app versions list --format="value(version.id)" --sort-by="~version.createTime" | tail -n +6 )
    gcloud app versions delete --quiet $VERSIONS

Answer 6

The command below may work:

gcloud app deploy app.yaml worker.yaml --promote --stop-previous-version

see: https://cloud.google.com/sdk/gcloud/reference/app/deploy

I cannot find any documents about the default values of promote and stop-previous-version. As far as I observed, promote is True and stop-previous-version is False though. I wrote both options for safety.

Answer 7

To turn off the instances associated with older versions you could try to use the teardown scripts (eventually customised) mentioned in the delete tutorial resources doc:

If you tried the tutorial about how to run Ruby on Compute Engine, see the Deployment and teardown scripts section of that tutorial

Careful with deleting the app, though, you can't re-use the app id!

Since the older versions are no longer the default ones at least no new instances will be automatically starting for them. You may still need to delete them manually, unless Google automatically deletes the older automatically assigned versions exceeding the max number of versions allowed for an app.

Answer 8

We created a little Python script to delete old versions automagically. We run it before every deploy so that our deploys always succeed (and never fail due to having too many versions deployed already).

https://pypi.org/project/appengine-clean/

Answer 9

Using zx, you can write it with relatively clean code.

#!/usr/bin/env zx

// firstly, deploy
await $`gcloud app deploy --quiet --project=XXX`

// secondly, delete versions except the one deployed just now
let versions = await $`gcloud app versions list --project=XXX | sed 's/  */:/g' | cut -f 2 -d : | tail -n +2 | ghead -n -1`
console.log("---")

versions = versions.toString().split("\n") // makes array
versions.pop() // removes last blank element created by new line
console.log(versions)

await Promise.all(versions.map(version =>
    $`gcloud app versions delete ${version} --project=XXX`
))
console.log("done!")

Answer 10

If you need to delete all of the versions except the active version's instances, you can use these commands:

versions_with_traffic=$(gcloud app versions list --hide-no-traffic --format="value(version.id)")

versions_to_be_deleted=$(gcloud app versions list --format="value(version.id)" | grep -v $versions_with_traffic)

gcloud app versions delete $versions_to_be_deleted --quiet

# if you need to delete multiple versions, then you need:
echo $versions_to_be_deleted | gcloud app versions delete --quiet

I believe the variable names are self-explanatory. However, if you still need help, please continue reading.

1. Get all of the version numbers with active traffic
2. Use grep -v to negate the original list, so that we can get the list of inactive version numbers
3. Delete the versions

You could also add the service=${service_name} filter in the command if you have multiple services.

For instance:

versions_with_traffic=$(gcloud app versions list --hide-no-traffic --service=default --format="value(version.id)")

versions_with_traffic=$(gcloud app versions list --hide-no-traffic --service=worker --format="value(version.id)")

Tips:

• You could add these scripts to your CI pipeline, so these gets cleaned up every deployment