0

I have a GAE app reading data (updated daily!) from the GCS bucket.

Problem: This app show the insights on the old data as it launches the instance already initialized earlier (currently active instance)

Question: How do I make sure that as the data is updated in the GCS bucket, GAE app reads the updated data? Maybe by initializing new instances of the app or any other alternative

App info:
Runtime: Standard python 3.7,
Scaling: automatic

Ismael Padilla
  • 5,246
  • 4
  • 23
  • 35
data_cruncher
  • 21
  • 4
  • What do you mean by "the instance already initialized earlier"? What does your GAE application do with the data in GCS? and when does it read from GCS? – Alex Oct 13 '20 at 15:55
  • Instance already initialized: Currently active instance, Role of data in GCS: used to obtain relevant insights/ visualization, Reading data from GCS: When the instance starts (when it receives loading request). This data read is initialized as global variable – data_cruncher Oct 13 '20 at 17:17

1 Answers1

1

So essentially you want a way to intentionally kill all existing instances of your GAE application everytime a GCS update goes out. I dont think there is a way to do that, at least not programmatically (you are able to manually delete instances from this GCP console page https://console.cloud.google.com/appengine/instances).

You could kind of do it by redeploying your app. CI/CD services are able to programmatically deploy to GAE, so you could have some daily cron job trigger a deploy.

However, the right way to do this would be to use some kind of persistent storage like Google Datastore instead of global variables on the instance. Then you could have a daily cron job that runs and downloads the GCS file and updates the data in datastore. Then all your instances of your application would be reading from datastore instead of global variables and would get the updates.

Alex
  • 5,141
  • 12
  • 26
  • I realize that eventually Google Firestore or Datastore is the way to go. For now, I found a similar use-case and a way to programmatically delete the instances using App Engine Admin API as outlined here: https://stackoverflow.com/questions/55203095/how-to-restart-google-app-engine-standard-service – data_cruncher Oct 14 '20 at 12:44