9

Im querying google ads api and need to save results as json. What is the best way to convert GoogleAdsRow type into json?

The result of the google ads appi look like this (the campaign and customer ids are fake):

campaign {
  resource_name: "customers/752830100/campaigns/22837002"
  id {
    value: 22837002
  }
  name {
    value: "test"
  }
}
metrics {
  clicks {
    value: 51
  }
  impressions {
    value: 33
  }
}

type = <class 'google.ads.googleads_v1.types.GoogleAdsRow'>

user912830823
  • 1,179
  • 3
  • 14
  • 25

5 Answers5

11

Google.protobuf has a method called json_format which can do exactly that. Below is a code example to help:

# Import the method 
from google.protobuf import json_format
import json

# Query the Ads API, 
ga_service = client.get_service('GoogleAdsService', version='YOUR_VERSION')
query = ('YOUR_QUERY')

# Response as an iterator made up of GoogleAdsRow  
response = ga_service.search(customer_id, query)

# Converting each of the GoogleAdsRow to json
for row in response : 
    json_str = json_format.MessageToJson(row)
    d = json.loads(json_str)
Matthieu
  • 316
  • 4
  • 14
  • 2
    Amazing, thanks for the share! And in case anyone ended up here because they wanted to convert google ads api result to a dataframe, you just need to add the following line to the above: df = pd.json_normalize(data=d['results']) – Saint Dec 11 '20 at 14:34
  • 4
    This worked for me but with the addition of `._pb` to my `row` object, e.g. `MessageToJson(row._pb)`. Otherwise I was getting a "DESCRIPTOR" error message. See this answer: https://stackoverflow.com/a/68216092, section 3. – Jason R Stevens CFA Oct 12 '21 at 02:59
  • 3
    change `MessageToJson` to `MessageToDict` and you can skip the loading with json – Ivan Gonzalez May 03 '22 at 01:26
3

Right now I am also looking for the same problem. One workaround that I have found is that use a CustomEncoder in json.dumps. Here is a sample:

class CustomEncoder(json.JSONEncoder):
def default(self, obj):
    if isinstance(obj, (google.protobuf.wrappers_pb2.StringValue,
                        google.protobuf.wrappers_pb2.Int64Value, 
                        google.protobuf.wrappers_pb2.DoubleValue)):
        return obj.value

    elif isinstance(obj, google.protobuf.pyext._message.RepeatedCompositeContainer):
        data = []
        try:
            while True:
                item = obj.pop()
                data.append(self.default(item))
        except IndexError:
            return data

    elif isinstance(obj, google.ads.google_ads.v1.proto.common.custom_parameter_pb2.CustomParameter):
        return { 
                self.default(obj.key): self.default(obj.value) 
                }
return json.JSONEncoder.default(self, obj)

Use the above encoder in json.dumps(data, cls=CustomEncoder)

This is the only solution I have so far come with. Will update it if I find a better solution.

Edited: Found the Solution. Here is the New Encoder class.

class GoogleProtoEncoder(json.JSONEncoder):
"""
Custom JSON Encoder for GoogleAdsRow. 

Usage: json.dumps(data, cls=GoogleProtoEncoder)
"""
def default(self, obj):
    """
    Overriden method. When json.dumps() is called, it actually calls this method if 
    this class is specified as the encoder in json.dumps().
    """
    if isinstance(obj, google.protobuf.message.Message) and hasattr(obj, 'value'):
        # This covers native data types such as string, int, float etc
        return obj.value

    elif isinstance(obj, google.protobuf.pyext._message.RepeatedCompositeContainer):
        # This is basically for python list and tuples
        data = []
        try:
            while True:
                item = obj.pop()
                data.append(self.default(item))
        except IndexError:
            return data

    elif isinstance(obj, google.ads.google_ads.v1.proto.common.custom_parameter_pb2.CustomParameter):
        # Equivalent to python dictionary
        return { 
                self.default(obj.key): self.default(obj.value) 
                }

    elif isinstance(obj, google.protobuf.message.Message):
        # All the other wrapper objects which can have different fields.
        return {key[0].name: getattr(obj, key[0].name) for key in obj.ListFields()}

    return json.JSONEncoder.default(self, obj)

Thanks.

Edited: Updated solution. Works in V7

import proto

response = ga_service.search_stream(search_request)
for batch in response:
    for row in batch.results:
        logging.debug(proto.Message.to_dict(row))
  • This works great when using previous version of the API. Do you know what has changed in V3? What needs to be updated? – Dave Davis Jul 10 '20 at 22:45
  • Hi, I ran into an issue, using the code. Although it works for the vast majority of the time I ran into an issue. Im running v5 as they now allow retrieval of discovery campaigns. I ran into this issue in one of my extractions `TypeError: Object of type RepeatedScalarContainer is not JSON serializable` – spak Sep 28 '20 at 15:26
  • @DaveDavis @spak Apologies.. I haven't looked into the newer versions.. I would recommend you to check the structure of `RepeatedScalarContainer` and add another `elif` statement to convert that accordingly. – Rahul Kumar Oct 05 '20 at 06:45
2

If you don't have to use the SDK, another option to get the json, is to query the API sending a post request directly, which returns a json response:

r = requests.post('https://googleads.googleapis.com/v3/customers/YOUR_CUSTOMER_ID/googleAds:searchStream', 
                 headers={'Authorization': f'Bearer {access_token}',
                         'developer-token' : developer_token,
                         'login-customer-id' : 'YOUR_CUSTOMER_ID',
                         'Content-Type': 'application/json'},
                params={"query" : query})
scott_m
  • 151
  • 2
  • 11
2

You can use the custom protobuf helper methods now as outlined here: https://developers.google.com/google-ads/api/docs/client-libs/python/library-version-10#wrapped_versus_native_protobuf_messages

campaign = client.get_type("Campaign")
json = type(campaign).to_json(campaign)
campaign = type(campaign).from_json(json)
Dave Davis
  • 844
  • 1
  • 9
  • 19
1

Step 1: Import this lib

from google.ads.googleads.errors import GoogleAdsException

Step 2: Send request

keyword_ideas = keyword_plan_idea_service.generate_keyword_ideas(
    request=request
)

Step 3: Convert response to json

keyword_ideas_json = MessageToDict(keyword_ideas)

Step 4: Do whatever you want with that json

print(keyword_ideas_json)

Note: This might though an attribute error: "DESCRIPTOR" error, so look at this answer: here

Rohit Nishad
  • 2,570
  • 2
  • 22
  • 32