2

I need to delete a BigQuery table partition using a Python 3 app running on the App Engine in the standard environment. It appears that the way to do this is through an HTTPS call, with an authentication token: https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/delete

https://googleapis.dev/python/google-api-core/latest/auth.html explains how to create credentials for the App Engine:

from google.auth import app_engine
credentials = app_engine.Credentials()

But as reported and explained in GCP-The App Engine APIs are not available, with py 3 and How to get credentials in Google AppEngine Python37 , trying to access app_engine in the Python 3 standard environment results in the following error

The App Engine APIs are not available

since the python37 runtime doesn't include proprietary app engine APIs such as the App Identity API.

Is there any way in which I can obtain a token from the App Engine for use with the HTTPS call to BigQuery? Or can a BigQuery partition be deleted by another means?

gimbal
  • 63
  • 8
  • In here (https://cloud.google.com/bigquery/docs/managing-tables#deleting_tables) you can see the ways to delete a table, might help you in case you would prefer other methods. – Stefan Neacsu Apr 07 '20 at 08:09
  • Thanks @Stefan Neascu. Since the problem is to do with deleting a partition in a partitioned table, rather than deleting the table itself, the only option seems to be the command line tool's `bq rm` command or an HTTP(S) call with the tables.delete API method: https://cloud.google.com/bigquery/docs/managing-partitioned-tables . Since your suggestion, I've tried running the `bq` command in a sub-process, but unfortunately `bq` doesn't seem to be available when running the Python code as a service on the App engine. – gimbal Apr 11 '20 at 14:05
  • See [this](https://googleapis.dev/python/bigquery/latest/generated/google.cloud.bigquery.client.Client.html#google.cloud.bigquery.client.Client), [this](https://stackoverflow.com/a/58992933/11135166), and [this](https://cloud.google.com/appengine/docs/standard/python3/access-control#apps). I hope this helps you. – asbovelw Apr 17 '20 at 15:57
  • Thanks @asbovelw. Your first link made me have a closer look at the google.cloud.bigquery.client.Client code. I found there was a protected member called _credentials, so client._credentials.token gave me the token I needed and the partition deletion request worked :-) . This avoided having to generate and store a JSON key file as per your third link. A downside though with the client._credentials.token approach is accessing a protected member rather than a public interface, so there is a risk that future changes to the bq library might stop it working. However, it works for now. Many thanks. – gimbal Apr 18 '20 at 21:29

0 Answers0