Disabling Billing on Google Cloud Using Google Cloud Function (Keyerror: 'data')

Question

I am attempting to write a Google Cloud Function to set caps to disable usage above a certain limit. I followed the instructions here: https://cloud.google.com/billing/docs/how-to/notify#cap_disable_billing_to_stop_usage.

This is what my cloud function looks like (I am just copying and pasting from the Google Cloud docs page linked above):

import base64
import json
import os
from googleapiclient import discovery
from oauth2client.client import GoogleCredentials
PROJECT_ID = os.getenv('GCP_PROJECT')
PROJECT_NAME = f'projects/{PROJECT_ID}'
def stop_billing(data, context):
    pubsub_data = base64.b64decode(data['data']).decode('utf-8')
    pubsub_json = json.loads(pubsub_data)
    cost_amount = pubsub_json['costAmount']
    budget_amount = pubsub_json['budgetAmount']
    if cost_amount <= budget_amount:
        print(f'No action necessary. (Current cost: {cost_amount})')
        return

    billing = discovery.build(
        'cloudbilling',
        'v1',
        cache_discovery=False,
        credentials=GoogleCredentials.get_application_default()
    )

    projects = billing.projects()

    if __is_billing_enabled(PROJECT_NAME, projects):
        print(__disable_billing_for_project(PROJECT_NAME, projects))
    else:
        print('Billing already disabled')


def __is_billing_enabled(project_name, projects):
    """
    Determine whether billing is enabled for a project
    @param {string} project_name Name of project to check if billing is enabled
    @return {bool} Whether project has billing enabled or not
    """
    res = projects.getBillingInfo(name=project_name).execute()
    return res['billingEnabled']


def __disable_billing_for_project(project_name, projects):
    """
    Disable billing for a project by removing its billing account
    @param {string} project_name Name of project disable billing on
    @return {string} Text containing response from disabling billing
    """
    body = {'billingAccountName': ''}  # Disable billing
    res = projects.updateBillingInfo(name=project_name, body=body).execute()
    print(f'Billing disabled: {json.dumps(res)}')

Also attaching screenshot of what it looks like on Google Cloud Function UI:

I'm also attaching a screenshot to show that I copied and pasted the relevant things to the requirements.txt file as well.

But when I go to test the code, it gives me an error:

Expand all | Collapse all{
 insertId: "000000-69dce50a-e079-45ed-b949-a241c97fdfe4"  
 labels: {…}  
 logName: "projects/stanford-cs-231n/logs/cloudfunctions.googleapis.com%2Fcloud-functions"  
 receiveTimestamp: "2020-02-06T16:24:26.800908134Z"  
 resource: {…}  
 severity: "ERROR"  
 textPayload: "Traceback (most recent call last):
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 383, in run_background_function
    _function_handler.invoke_user_function(event_object)
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 217, in invoke_user_function
    return call_user_function(request_or_event)
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 214, in call_user_function
    event_context.Context(**request_or_event.context))
  File "/user_code/main.py", line 9, in stop_billing
    pubsub_data = base64.b64decode(data['data']).decode('utf-8')
KeyError: 'data'
"  
 timestamp: "2020-02-06T16:24:25.411Z"  
 trace: "projects/stanford-cs-231n/traces/8e106d5ab629141d5d91b6b68fb30c82"  
}

Any idea why?

Relevant Stack Overflow Post: https://stackoverflow.com/a/58673874/3507127

Your problem is with the code that calls `stop_billing`. You are trying to access a key `data` that is not in the dictionary `data` on line 9. Since that code is not present in your question, an answer is difficult. Note: Do not show your code as a picture. Edit your question and include the code as a formatted code block. — John Hanley, Feb 06 '20 at 17:35
It looks like your function is invoked by Pub/Sub. The first parameter is `event`. This normally has the key `data`. I would log the dictionary to Stackdriver to debug what you are actually receiving. Also, I would add code like this to prevent crashes `if 'data' in 'data':`. Rename `data` back to `event` to make your variable names mean something. — John Hanley, Feb 06 '20 at 17:40
@JohnHanley -- thanks, will give that a try. Edited the original post to show the actual code; however, I just copied and pasted from the Google Cloud docs: https://cloud.google.com/billing/docs/how-to/notify#cap_disable_billing_to_stop_usage (I didn't make any changes) — Vincent, Feb 06 '20 at 17:45
I tried setting this up and I had no issues with it, did you add the requirements to the requirements.txt files for the function as mentioned at the docs? — rsalinas, Feb 07 '20 at 13:43
Yes, I did! Edited the original post to show screenshot of my requirements.txt file as well. Did you just strictly copy and paste or did you edit anything else? — Vincent, Feb 07 '20 at 21:29
How did you test your code? If you just clicked "TEST THIS FUNCTION" on the console, it was likely that you didn't supply some JSON data as the "data" parameter. The way I used to test this function is to edit the code so that the function is triggered at a very low threshold, such as when the cost is over 5% of the budget, and see if the billing would be stopped automatically. Of course, you'd have to plan this carefully to not disrupt any important VMs. — Yan Li, Feb 08 '20 at 04:36
@YanLi -- thanks! I was just testing the function by clicking "Test this function". I will do what you suggested which is set the budget to be really low and see if the VM shuts down automatically. — Vincent, Feb 10 '20 at 17:12
@YanLi Hmm. I've waited a couple days, and it looks like the function has not run. How do I get it to run? — Vincent, Feb 12 '20 at 16:31
You should be able to see in Stackdriver the log for your function, if it was executed as well as it's response status or if it had any issue — rsalinas, Feb 14 '20 at 14:53

Frits · Answer 1 · 2020-10-12T10:40:00.363

There seems to be an error in the code Google provided. I got it working when I changed the stop_billing function:

def stop_billing(data, context):
    if 'data' in data.keys():
        pubsub_data = base64.b64decode(data['data']).decode('utf-8')
        pubsub_json = json.loads(pubsub_data)
        cost_amount = pubsub_json['costAmount']
        budget_amount = pubsub_json['budgetAmount']
    else:
        cost_amount = data['costAmount']
        budget_amount = data['budgetAmount']
    if cost_amount <= budget_amount:
        print(f'No action necessary. (Current cost: {cost_amount})')
        return    
    if PROJECT_ID is None:
        print('No project specified with environment variable')
        return
    billing = discovery.build('cloudbilling', 'v1', cache_discovery=False, )
    projects = billing.projects()
    billing_enabled = __is_billing_enabled(PROJECT_NAME, projects)
    if billing_enabled:
        __disable_billing_for_project(PROJECT_NAME, projects)
    else:
        print('Billing already disabled')

The problem is that the pub/sub message provides input as a json message with a 'data' entry that is base64 encoded. In the testing functionality you provide the json entry without a 'data' key and without encoding it. This is checked for in the function that I rewrote above.

Disabling Billing on Google Cloud Using Google Cloud Function (Keyerror: 'data')

1 Answers1

Linked