A dictionary value modified inside the scope of a function is also modified outside, without using globals

Question

I will not provide a reproducible example, as for this specific problem I can not reproduce the problem with a small dummy example. Yet, I believe something weird happened, something which is not a simple typo/mistake on my side. Maybe this problem will ring a bell, and someone might have an explanation.

Let's consider 3 files of my project:

cli/main.py
main.py
nfb.py

In main.py, a settings dictionary is loaded from a .json file. One of the specific value in this dictionary is settings = {'ONLINE':{'APPLY_REJECTION': True}}. It then calls the main function from the cli/main.py file which argument is the settings dictionary.

main.py

from cli.main import main 

settings = load_settings(fname)
main(settings)

As you might guess from the acronym CLI, the function called is a simple command-line interface where the user is provided with X possibilities (e.g. 9 with keys from 1 to 9) and based on the input, another function is called. The input phase is then repeated until an exit key, e.g. 0, is pressed.

cli/main.py

def main(settings):
    while True:
        selection = input_menu(options)
        if selection == 0:
            break # exit key

        elif selection == 1
            x, y = nfb.run(a, b, c, d, settings)

This is very schematic. Obviously in my code, there is error checking on the input, the variables are defined, ... One point to note however, the dictionary settings is passed to the run function from the nfb.py file.

In nfb.py, the function run might modify the variable settings['ONLINE']['APPLY_REJECTION'] from True to False. Once, the function is exited and the program is waiting for a new user input (selection), as we have left the scope of nfb.run(), the variable settings['ONLINE']['APPLY_REJECTION'] should be back to True for future nfb.run() calls.

However, this is not what I observed. The variable settings['ONLINE']['APPLY_REJECTION'] stayed to False, outside the scope of nfb.run().

Even more surprising, I tried doing this in nfb.py:

import copy

def run(a, b, c, d, settings):
    backup_settings = copy.deepcopy(settings)
    
    # do stuff
    if condition:
        settings['ONLINE']['APPLY_REJECTION'] = False

    # restore settings
    settings = backup_settings

With a few prints, I could confirm that before the # restore settings, the variable settings['ONLINE']['APPLY_REJECTION'] was set to False, and after settings = backup_settings, the last line of the function, the variable settings['ONLINE']['APPLY_REJECTION'] was set to True.

However, if in the while loop from the CLI I print the same variable before or right after the selection (i.e. user input), I could see that this variable remained set to False after being set to False once in the nfb.run() function, and this despite the backup/restoration lines!

cli/main.py

def main(settings):
    while True:
        print (settings['ONLINE']['APPLY_REJECTION'])
        selection = input_menu(options)
        if selection == 0:
            break # exit key

        elif selection == 1
            x, y = nfb.run(a, b, c, d, settings)

The above piece of code will print True the first time, and False once the variable has been set to False once by nfb.run().

To note, there is absolutely nowhere else where this variable is modified. Moreover, my solution to the problem was to extract the variable from the dictionary at the beginning of run, and then work with this extract variable, thus avoiding to change anything from the settings dictionary.

Workaround:

def run(a, b, c, d, settings):
    apply_rejection = settings['ONLINE']['APPLY_REJECTION']
    
    # do stuff
    if condition:
        apply_rejection = False

This workaround works (nothing else was changed!) and shows that the modification applied to this variable settings['ONLINE']['APPLY_REJECTION'] was the one in nfb.run() and that as I claim, there is no other line meddling with the setting dictionary directly anywhere.

Discussion:

It looks to me like in my initial approach, the settings dictionary inside nfb.run() pointed to the same object as to the one outside, loaded as parameter of the cli.main(settings) function. Even more surprising, this link seemed to be broke as soon as the variable settings inside nfb.run() is re-assigned with a new value (in my case the deepcopy of the original object).

Does anyone who managed to follow this long post have an idea, an explanation, about this behavior?

I admit I read just the title and discussion, BUT are you aware that some containers like dict, list are mutable? — buran, Apr 15 '21 at 14:32
@buran I am but I might not grasp the full extent of this notion. A dict is mutable as you can add new key/values to it. However, those modification should be limited to the scope of the function you are in. When I test a very simple function taking a dict in input, change one of the values from True to False, print inside the function the result and outside, the outside dictionary is not modified. In my case it was, and that is what I do not understand. — Mathieu, Apr 15 '21 at 14:36
_When I test a very simple function taking a dict in input, change one of the values from True to False, print inside the function the result and outside, the outside dictionary is not modified_ - that is not correct — buran, Apr 15 '21 at 14:40
@buran Not really as the example given is obvious to me and looks different from my case. I will add to the discussion a small piece of code which shows the expected behavior (working) and which is not the behavior I observed in my program. — Mathieu, Apr 15 '21 at 14:40
@buran Indeed that is not correct. Problem solved I am a complete idiot! — Mathieu, Apr 15 '21 at 14:42

score 1 · Accepted Answer · answered Apr 15 '21 at 14:42

1

spam = {'foo':'bar'}

def eggs(arg):
    arg['foo'] = 'baz'

print(spam)
eggs(spam)
print(spam)

output

{'foo': 'bar'}
{'foo': 'baz'}

dict is mutable type. The link I shared explains exactly that, although the example there is with different mutable type - list.

answered Apr 15 '21 at 14:42

buran

13,682
10
36
61

Yes, when you commented that this is not correct I retested this too. Dunno what I did last time, most likely forgot the call to `egg` and thought I was getting a different behavior.. Stupid mistake on my side which I manage to do several times as I did this little test several times.. – Mathieu Apr 15 '21 at 14:43

A dictionary value modified inside the scope of a function is also modified outside, without using globals

1 Answers1