There's a way to get all the results into a DataFrame with for?

Question

I have a list containing customer ids:

clients = ['act_1078577379193828', 'act_1503882476478990', 'act_232830897389702']

I'm running an asynchronous job in Python that returns me campaign data from customers inside a For:

for advertiser in clients:
    async_job = AdAccount(advertiser).get_insights(fields=fields, params=params, is_async=True)
    async_job.api_get()
    while async_job[AdReportRun.Field.async_status]!= 'Job Completed':
        time.sleep(1)
        async_job.api_get()
    time.sleep(1)
    df = pd.DataFrame(async_job.get_result())

I am trying to create a DataFrame containing the data from all the customer ids in the list, but when I create it, it only brings in data from one customer.

I was able to get the results from the other customers using If:

for advertiser in clients:
    async_job = AdAccount(advertiser).get_insights(fields=fields, params=params, is_async=True)
    async_job.api_get()
    while async_job[AdReportRun.Field.async_status]!= 'Job Completed':
        time.sleep(1)
        async_job.api_get()
    time.sleep(1)
    if advertiser == 'act_1078577379193828':
        df1 = pd.DataFrame(async_job.get_result())
    elif advertiser == 'act_1503882476478990':
        df2 = pd.DataFrame(async_job.get_result())
    elif advertiser == 'act_232830897389702':
        df3 = pd.DataFrame(async_job.get_result())

But since I'm going to use a list with more than 44 customer ids, I feel it gets very poorly optimized this way.

Do you know of another way to put all the data together in a single dataframe without so many if?

*async_job.get_result() it's where returns the data.

Output from async_job.get_result():

[<AdsInsights> {
    "account_id": "232830897389702",
    "account_name": "Advertiser_Account_Name",
    "actions": [
        {
            "action_type": "link_click",
            "value": "4"
        },
        {
            "action_type": "post_engagement",
            "value": "4"
        },
        {
            "action_type": "page_engagement",
            "value": "4"
        }
    ],
    "ad_name": "***",
    "adset_name": "***",
    "campaign_name": "***",
    "clicks": "15",
    "cpc": "0.568",
    "cpm": "15.160142",
    "ctr": "2.669039",
    "date_start": "2022-08-26",
    "date_stop": "2022-08-26",
    "frequency": "1.05838",
    "impressions": "562",
    "inline_post_engagement": "4",
    "objective": "LEAD_GENERATION",
    "reach": "531",
    "spend": "8.52"
},

What are the values of `async_job.get_result()` for each loop? I'm pretty sure I have an answer, but I want to confirm cause the `data` parameter of the `DataFrame` constructor takes a lot of different forms and I'm not sure if there might be edge cases. For more info, please read [mre] and for specifics, see [How to make good reproducible pandas examples](/q/20109391/4518341). — wjandrea, Sep 28 '22 at 18:56
@wjandrea Sorry, I didn't put the output of `async_job`. Note that "account_id" it's the same from client's list. — André Filho, Sep 28 '22 at 19:06
Hmm, that repr is not valid Python data. Could it serialize somehow, like to a list of dicts? — wjandrea, Sep 28 '22 at 19:12

score 1 · Accepted Answer · answered Sep 28 '22 at 23:41

IIUC, this should work...

dfs = []
for advertiser in clients:
    async_job = AdAccount(advertiser).get_insights(fields=fields, params=params, is_async=True)
    async_job.api_get()
    while async_job[AdReportRun.Field.async_status]!= 'Job Completed':
        time.sleep(1)
        async_job.api_get()
    time.sleep(1)
    df = pd.DataFrame(async_job.get_result())
    dfs.append(df)

df = pd.concat(dfs)

There's a way to get all the results into a DataFrame with for?

1 Answers1