Python: An append() is causing an infinite loop

Question

I have some campaigns that I want to modify but I've to call the server and export all the campaign settings modify or add the information I want and send it back to the server.

My modifications are stored in an SQLite db, so first, I query the db looking for the account IDs (I can have many campaigns that are in different accounts), then I look for campaigns per account.

theCursor.execute('SELECT * FROM SandboxPubTB') 
rows = theCursor.fetchall()

for row in rows:
    accounts.append(row[7])

for account in set(accounts):
    theCursor.execute('SELECT CAMPAIGNNAME, CAMPAIGNID FROM SandboxPubTB WHERE ACCOUNTID =?', (account,) ) 
    campaignRows = theCursor.fetchall()
    for campaign in campaignRows:
        campId= campaign[0] + "," + campaign[1] + "," + account
        campaigns.append(campId)

My campId list contains the campaign name, campaign ID and the campaigns account ID in each entry.

After that, I iterate through the list and assign the values to a variable at the campaign level:

for campaignNameId in set(campaigns):
    campaignNameId = campaignNameId.split(',')
    campaignName = campaignNameId[0]
    campaignId = campaignNameId[1]
    campaignAccount = campaignNameId[2]

Now it's time to call the server to export the campaign settings:

    targetUrl = "https://sites.com/api/1.0/" + campaignAccount + "/campaigns/"+ campaignId
    resp = requests.get(url=targetUrl, headers=header)

    r = resp.json()

    temp = r['publisher_bid_modifier']['values']

If I print temp this is what I get:

[{
    'target': 'msn-can',
    'cpc_modification': 0.5
}, {
    'target': 'msn-can-home',
    'cpc_modification': 0.5
}, {
    'target': 'smartify-journalistatecom',
    'cpc_modification': 0.5
}, {
    'target': 'foxnews-iosapp',
    'cpc_modification': 1.22
}]

And finally, here is where I'm stuck. What I'm trying to do is iterate through the dictionary list above and if "target" value exists and "cpc_modification" is different than what I've in the db I'll change the cpc value. If "target" doesn't exist I want to append "target" and "cpc_modification" to the dictionary list.

I succeeded with the first part, but the append part is different. In the elif, even if I use an else, the temp.append triggers an infinite loop and I have no idea why.

    for dt in r['publisher_bid_modifier']['values']:
        #print(dt['target'])
        #if dt['target']:
        theCursor.execute('SELECT ADJUST, SITE FROM SandboxPubTB WHERE CAMPAIGNNAME =?', (campaignName,) ) 
        campaignRows = theCursor.fetchall()
        for t in campaignRows:
            if t[1] == dt['target'] :
                dt['cpc_modification'] = "{:.2f}".format((int(t[0]) / 100) + 1)
            elif dt['target'] not in temp:
                temp.append("{'target': "'" + t[1] + "'", 'cpc_modification': "'" + str(t[0]) + "'"}")

That's weird because I've tried to emulate the same behavior with a local variable and it seems to work without problem.

data = [{
                "target": "publisher1",
                "cpc_modification": 1.5
            },
            {
                "target": "publisher2",
                "cpc_modification": 0.9
            }
        ]

for t in data:
    if t['target'] == "publisher10":
        t['cpc_modification'] = 1.9

    else:
        data.append({'target': 'publisher10', 'cpc_modification': 12})


    print(t['target'], t['cpc_modification'])

print(data)

I've tried many things, and I can't figure out what is wrong.

Can you reduce this to a minimal example? I'm not quite getting which part of the code isn't doing what you expect, and in which way. — mkrieger1, Jun 07 '19 at 20:10
`temp` and `dt` point to the same object. Thus you are iterating and changing an object at the same time. You need `for dt in r['publisher_bid_modifier']['values'].copy():` or `from copy import deepcopy` & `for dt in deepcopy(r['publisher_bid_modifier']['values']:)` — Error - Syntactical Remorse, Jun 07 '19 at 20:11
Possible duplicate of [Python: Adding element to list while iterating](https://stackoverflow.com/questions/3752618/python-adding-element-to-list-while-iterating) — pault, Jun 07 '19 at 20:12
It would seem that you're modifying (inplace) the contents of the iterable: `r['publisher_bid_modifier']['values']`. note that `temp = r['publisher_bid_modifier']['values']` just maps the *name* `temp` to that object. You're modifying the same thing over which you're iterating in `for dt in ...`. — David Zemens, Jun 07 '19 at 20:13
FTR, the first comments refers to [How to create a Minimal, Reproducible Example](https://stackoverflow.com/help/mcve) — Aprillion, Jun 07 '19 at 20:17
I thought that by assigning a variable to another variable it would create a new one. Thank you for your help. For the duplicate, my first question was about the problem in it's entirity and I didn't get the answer that would solve that problem so in this one I tried to point out where my problem is being sure that if find a solution to that particular issue it would solve my issue in general. — Creek Barbara, Jun 07 '19 at 20:21

score 1 · Accepted Answer · answered Jun 07 '19 at 20:18

I believe your problem is in here. You are iterating over 'data' but you are also appending stuff to 'data' as you are iterating over it. You should create a new variable to append to.

temp = data
for t in data:
    if t['target'] == "publisher10":
        t['cpc_modification'] = 1.9

    else:
        temp.append({'target': 'publisher10', 'cpc_modification': 12})


    print(t['target'], t['cpc_modification'])

print(temp)

I made an example here of what I believe is happending

Python: An append() is causing an infinite loop

1 Answers1