Constructor parameters re-used for new object?

Question

I am new to Python and while using it to write a small application I encountered a problem. As I can't find anything simliar on Google, I might have missunderstood some basic concepts of Python. It would be great if someone of you guys can explain that to me.

So here is a minimal working example. Please ignore the class and variable names, that is as said just an artificial example, but it shows what I mean. You can c&p it to your IDE and run it for yourself.

class State(object):
    def __init__(self, wrappers = []):
        self.wrappers = wrappers

    def add(self, name, items):
        lst = KeyValList(name)
        for item in items:
            lst.add(item, items[item])
        self.wrappers.append(Wrapper(lst))

class Wrapper(object):
    def __init__(self, kv_lst):
        self.kv_lst = kv_lst

class KeyValList(object):
    def __init__(self, name, kvals = []):
        self.name = str(name)
        self.kvals = kvals

    def add(self, key, value):
        self.kvals.append(KeyVal(key, value))

class KeyVal(object):
    def __init__(self, key, val):
        self.key = str(key)
        self.val = val

Ok, that is the definition of the classes I use.
Right below I write the following:

state = State()
kv = { "key1" : "1", "key2" : "2", "key3" : "3" }
state.add("1st", kv)
kv = { "keyX" : "X", "keyY" : "Y", "keyZ" : "Z" }
state.add("2nd", kv)

for wrap in state.wrappers:
    print wrap.kv_lst.name, "-->",
    for kv in wrap.kv_lst.kvals:
        print "%s:%s" % (kv.key, kv.val),
    print ""

As a Java programmer I would expect the following output:

1st --> key3:3 key2:2 key1:1
2nd --> keyZ:Z keyY:Y keyX:X

However, with this program in Python I get the following output instead:

1st --> key3:3 key2:2 key1:1 keyZ:Z keyY:Y keyX:X 
2nd --> key3:3 key2:2 key1:1 keyZ:Z keyY:Y keyX:X 

Why do both lists contain all key/value pairs?

I expect the problem somewhere in the KeyValList class, because while debugging when the state.add() method is called in the main program for the 2nd time the kvals parameter of the KeyValList constructor contains the former pairs. But how can that be? I do not provide anything to this kvals parameter, shouldn't it therefore be initialised to an empty list, due to the parameters default value?

PS: I am using Python 2.7.3 on Windows.

Answer 1

This is basically an instance of a very common -- and rather counterintuitive at first -- feature of Python, the mutable default argument. The gist is that default arguments are properties of the function object, and thus if you mutate them in one function call you mutate them for all function calls.

This example is a little more complicated than the standard mutable default argument issue, because you're also passing the same object around as an attribute of and instance of the class. But there's only one list object in your code -- the one created by [] -- and hence mutating it via the instance attribute is the same as mutating it on the function object.

In other words, when you write

self.kvals.append(...)

you're appending some data to a particular list object. Which list object? The one created by the [] in the definition of the function, and thus the same one that Python is using as the default argument to the function.

Answer 2

Instead of using mutable default arguments, use None as a placeholder and have your functions/methods construct new containers on each call.

Change this:

class State(object):
    def __init__(self, wrappers = []):
        self.wrappers = wrappers

To this:

class State(object):
    def __init__(self, wrappers = None):
        if wrappers is None:
           wrappers = []
        self.wrappers = wrappers

There is a nice explanation of the issue in this blogpost.