4

I have a django repository setup in gitlab and I am trying to automate build and deploy on google cloud using gitlab CI/CD.

The app has to be deployed on App Engine and has to use CloudSQL for dynamic data storage.

Issue that I am facing is while executing migration on the db, before deploying my application.

I am supposed to run ./manage.py migrate which connects to cloudSQL.

I have read that we can use cloud proxy to connect to cloudSQL and migrate db. But it kind of seems like a hack. Is there a way to migrate my db via CI/CD pipeline script?

Any help is appreciated. Thanks.

Saket Patel
  • 413
  • 3
  • 13
  • If you want to use cloudBuild and Github and public cloud sql instance then refer - [link](https://stackoverflow.com/questions/68090672/deploy-django-to-gae-standard-from-cloud-build?noredirect=1#comment120349498_68090672) – Aseem Jun 26 '21 at 20:23

2 Answers2

5

When running Django in the App Engine Standard environment the recommended way of approaching database migration is to run ./manage.py migrate directly from the Console shell or from your local machine (which requires using the cloud sql proxy).

If you want the database migration to be decoupled from your application deployment and run it in Gitlab CI/CD you could do something along these lines:

  • Use as base image google/cloud-sdk:latest
  • Acquire credentials gcloud auth activate-service-account --key-file $GOOGLE_SERVICE_ACCOUNT_FILE
  • Download the cloudsqlproxy with wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 -O cloud_sql_proxy and make it executable chmod +x cloud_sql_proxy.
  • Start the proxy ./cloud_sql_proxy -instances="[YOUR_INSTANCE_CONNECTION_NAME]"=tcp:3306.
  • Finally, run the migration script.

You might also create a custom docker image that already does what's above behind the scenes, the result would be the same.

If you want to read further on the matter I suggest taking a look at the following articles link1 link2.

Happy-Monad
  • 1,962
  • 1
  • 6
  • 13
  • Starting the proxy occupies the terminal though - how would you run it in the background, to allow you to continue to the migration step? – Tielman Nieuwoudt Jan 18 '21 at 21:32
  • @TielmanNieuwoudt You may open a new Cloud Shell tab (Or terminal in a VM/ on-prem machine) and run the migration script therein. – Happy-Monad Jan 19 '21 at 08:34
  • Thanks for the suggestion. I'm focusing on the fact that it's in a CI/CD script. So whether it's on-premise or in something like CircleCI, I'm trying to find a way to start the proxy in a way that doesn't block the terminal, without user interaction. – Tielman Nieuwoudt Jan 19 '21 at 12:09
  • To avoid blockage of terminal run cloud proxy command in background mode. In unix system command followed by & symbol will run a command in background mode – praveen Aug 12 '21 at 06:25
0

I'm also trying to find the correct way to do that. One other hacky way would be to just add a call to it in the settings file that is loaded with your app. something like the migrate.py file does:

from django.core.management import execute_from_command_line
execute_from_command_line(['./manage.py', 'migrate'])

so everytime after you'll deploy a new version of the app it will also run the migrate.

I want to beleive there are other ways, not involving the proxy, especially if you also want to work with a private ip for the sql - then this script must run in the same vpc.

kalsky
  • 469
  • 4
  • 4
  • cloudbuild is a great tool to deploy . It has nice integration with Github, but it doesnt support VPC. So cannot create django migrations on private cloud sql instance. Did you find a better approach for private sql so far? – Aseem Jun 26 '21 at 20:22