0

I've recently started working with the Facebook Marketing API, using the facebook_business SDK for Python (running v3.9 on Ubuntu 20.04). I think I've mostly wrapped my head around how it works, however, I'm still kind of at a loss as to how I can handle the arbitrary way in which the API is rate-limited.

Specifically, what I'm attempting to do is to retrieve all Ad Sets from all the campaigns that have ever run on my ad account, regardless of whether their effective_status is ACTIVE, PAUSED, DELETED or ARCHIVED.

Hence, I pulled all the campaigns for my ad account. These are stored in a dict, whereby the key indicates the effective_status, like so, called output:

{'ACTIVE': ['******************',
  '******************',
  '******************'],
 'PAUSED': ['******************',
  '******************',
  '******************'}

Then, I'm trying to pull the Ad Set ids, like so:

import pandas as pd
import json
import re
import time
from random import *

from facebook_business.api import FacebookAdsApi
from facebook_business.adobjects.adaccount import AdAccount # account-level info
from facebook_business.adobjects.campaign import Campaign # campaign-level info
from facebook_business.adobjects.adset import AdSet # ad-set level info
from facebook_business.adobjects.ad import Ad # ad-level info

# auth init
app_id = open(APP_ID_PATH, 'r').read().splitlines()[0]
app_secret = open(APP_SECRET_PATH, 'r').read().splitlines()[0]
token = open(APP_ACCESS_TOKEN, 'r').read().splitlines()[0]

# init the connection
FacebookAdsApi.init(app_id, app_secret, token)

campaign_types = list(output.keys())

ad_sets = {}

for status in campaign_types:
    ad_sets_for_status = []
    for campaign_id in output[status]:
        # sleep and wait for a random time
        sleepy_time = uniform(1, 3)
        time.sleep(sleepy_time)
        # pull the ad_sets for this particular campaign
        campaign_ad_sets = Campaign(campaign_id).get_ad_sets()
        for entry in campaign_ad_sets:
            ad_sets_for_status.append(entry['id'])
    ad_sets[status] = ad_sets_for_status

Now, this crashes at different times whenever I run it, with the following error:

FacebookRequestError: 

  Message: Call was not successful
  Method:  GET
  Path:    https://graph.facebook.com/v11.0/23846914220310083/adsets
  Params:  {'summary': 'true'}

  Status:  400
  Response:
    {
      "error": {
        "message": "(#17) User request limit reached",
        "type": "OAuthException",
        "is_transient": true,
        "code": 17,
        "error_subcode": 2446079,
        "fbtrace_id": "***************"
      }
    }

I can't reproduce the time at which it crashes, however, it certainly doesn't take ~600 calls (see here: https://stackoverflow.com/a/29690316/5080858), and as you can see, I'm sleeping ahead of every API call. You might suggest that I should just call the get_ad_sets method on the AdAccount endpoint, however, this pulls fewer ad sets than the above code does, even before it crashes. For my use-case, it's important to pull ads that are long over as well as ads that are ongoing, hence it's important that I get as much data as possible.

I'm kind of annoyed with this -- seeing as we are paying for these ads to run, you'd think FB would make it as easy as possible to retrieve info on them via API, and not introduce API rate limits similar to those for valuable data one doesn't necessarily own.

Anyway, I'd appreciate any kind of advice or insights - perhaps there's also a much better way of doing this that I haven't considered.

Many thanks in advance!

nikUoM
  • 639
  • 1
  • 8
  • 18

1 Answers1

0

The error with 'code': 17 means that you reach the limit of call and in order to get more nodes you have to wait.

Firstly I would handle the error in this way:

from facebook_business.exceptions import FacebookRequestError

...

for status in campaign_types:
    ad_sets_for_status = []
    for campaign_id in output[status]:

        # keep trying until the request is ok
        while True:
            try:
                campaign_ad_sets = Campaign(campaign_id).get_ad_sets()
                break
            except FacebookRequestError as error:
                if error.api_error_code() in [17, 80000]:
                    time.sleep(sleepy_time) # sleep for a period of time
           
        for entry in campaign_ad_sets:
            ad_sets_for_status.append(entry['id'])
    ad_sets[status] = ad_sets_for_status

I'd like to suggest you moreover to fetch the list of nodes from the account (by using the 'level': node param in params) and by using the batch calls: I can assure you that this will help you a lot and it will decrease the program run time.

I hope I was helpful.

Sprizgola
  • 416
  • 2
  • 10