Deploy django to GAE standard from cloud build

Question

I run following bash commands from my local machine to deploy django project to App engine.

python manage.py migrate
python manage.py collectstatic --noinput
gsutil rsync -R static/ gs://xyz4/static
gcloud config set project project_name_1
gcloud app deploy --quiet

I would like to set it up on cloud build. I have enabled PUSH triggers on cloud build. Need help in creating cloudbuild.yaml file

Answer 1

CloudBuild doesnt support VPC hence cannot be used for migration to private cloud sql - link

Following are steps I use when deploying code from Github repo to App engine standard. Each step is dependant on previous step running successfully.

• Create python venv & install all dependencies
• Install gcloud proxy & make it executable
• Turn on proxy, activate venv, run tests, if test pass then make migrations, collect static files
• Upload static files to public bucket
• deploy code to GAE standard

cloudbuild.yaml:

  - id: setup-venv
    name: python:3.8-slim
    timeout: 100s
    entrypoint: sh
    args:
      - -c
      - '(python -m venv my_venv && . my_venv/bin/activate && pip install -r requirements.txt && ls)'
    waitFor: [ '-' ]

  - id: proxy-install
    name: 'alpine:3.10'
    entrypoint: sh
    args:
      - -c
      - 'wget -O /workspace/cloud_sql_proxy https://storage.googleapis.com/cloudsql-proxy/v1.21.0/cloud_sql_proxy.linux.amd64 &&  chmod +x /workspace/cloud_sql_proxy'
    waitFor: [ 'setup-venv' ]

  - id: run-tests-with-proxy
    name: python:3.8-slim
    entrypoint: sh
    args:
      - -c
      - '(/workspace/cloud_sql_proxy -dir=/workspace -instances="<instance_name>=tcp:3306" & sleep 2) && (. my_venv/bin/activate && python manage.py test --noinput && python manage.py migrate && python manage.py collectstatic --noinput )'
    waitFor: [ 'proxy-install' ]
    env:
      - 'CLOUD_BUILD=1'
      - 'PYTHONPATH=/workspace'

  # if tests fail, these sections wont execute coz they waitFor tests section
  - id: upload-static-to-bucket
    name: 'gcr.io/cloud-builders/gsutil'
    entrypoint: 'bash'
    args: [ '-c', 'gsutil rsync -R ./static/ gs://<bucket_name>/static' ]
    waitFor: [ 'run-tests-with-proxy' ]

  - id: deploy
    name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: 'bash'
    args: [ '-c', 'gcloud app deploy --quiet' ]
    waitFor: [ 'upload-static-to-bucket' ]

Scope for improvement:

• how to have args broken into multiple lines instead of everything being on one line
• If tests run on local postgres instance on cloudbuild instead of production cloud sql instance, that would be nice. I was able to create a postgres instance, but it did not run in background, hence when running tests my code could not connect to this local instance

postgres in cloudbuild.yaml:

- id: setup-postgres
  name: postgres
  timeout: 500s
  waitFor: [ '-' ]
  env:
    - 'POSTGRES_PASSWORD=password123'
    - 'POSTGRES_DB=aseem'
    - 'POSTGRES_USER=aseem'
    - 'PGPORT=5432'
    - 'PGHOST=127:0:0:1'
    

Answer 2

Cloud Build support to run on custom VPC with worker pools.

You can create a worker pool to access resources in VPC. If you will access when building you must click "Assign external IPs" in your worker pool settings and you mus use below code in your cloudbuild yaml:

options: 
  pool:
    name: "projects/${PROJECT_ID}/locations/${_REGION}/workerPools/${_WORKER_POOL}"

substitutions:
  _REGION: 'your_region'
  _WORKER_POOL: 'your_pool_name'

If you need to know your outbound ip you can access it from Cloud Nat and if you have firewall rules for outbound you have to assign a dedicated ip.