Why does initializing a variable via a python default variable keep state across object instantiation?

Question

I hit an interesting python bug today in which instantiating a class repeatedly appears to be holding state. In later instantiation calls the variable is already defined.

I boiled down the issue into the following class/shell interaction. I realize that this is not the best way to initialize a class variable, but it sure should not be behaving like this. Is this a true bug or is this a "feature"? :D

tester.py:

class Tester():
        def __init__(self):
                self.mydict = self.test()

        def test(self,out={}):
                key = "key"
                for i in ['a','b','c','d']:
                        if key in out:
                                out[key] += ','+i
                        else:   
                                out[key] = i 
                return out

Python prompt:

Python 2.6.6 (r266:84292, Oct  6 2010, 00:44:09) 
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
>>> import tester
>>> t = tester.Tester()
>>> print t.mydict
{'key': 'a,b,c,d'}
>>> t2 = tester.Tester()
>>> print t2.mydict
{'key': 'a,b,c,d,a,b,c,d'}

Dupe: http://stackoverflow.com/questions/1132941/least-astonishment-in-python-the-mutable-default-argument — bstpierre, Oct 08 '10 at 01:38
Could it be interpreting `out` as a global variable, thus keeps adding to it whenever you call `test`? — Assaf Lavie, Oct 08 '10 at 01:27
Possible duplicate of ["Least Astonishment" and the Mutable Default Argument](http://stackoverflow.com/questions/1132941/least-astonishment-and-the-mutable-default-argument) — David Cain, May 09 '17 at 14:04

score 15 · Accepted Answer · edited May 09 '17 at 14:15

It is a feature that pretty much all Python users run into once or twice. The main usage is for caches and the like to avoid repetitive lengthy calculations (simple memoizing, really), although I am sure people have found other uses for it.

The reason for this is that the def statement only gets executed once, which is when the function is defined. Thus the initializer value only gets created once. For a reference type (as opposed to an immutable type which cannot change) like a list or a dictionary, this ends up as a visible and surprising pitfall, whereas for value types, it goes unnoticed.

Usually, people work around it like this:

def test(a=None):
    if a is None:
        a = {}
    # ... etc.

Ya good explanation. I want to add another line. function is an object in python, when we call for the first time function object is created, after that we are reusing the same object. you can confirm by printing the id(Test) after every call. you will get the same function object id, which means only one object is sharing. — James Sapam, Nov 27 '13 at 02:12

score 5 · Answer 2 · edited May 23 '17 at 12:10

5

In general, default method arguments shouldn't be mutable. Instead do:

def test(self, out=None):
   out = out or {}
   # other code goes here.

See these links for more details on why this is necessary and why it's a "feature" of the python language rather than a bug.

edited May 23 '17 at 12:10

Community

1
1

answered Oct 08 '10 at 01:33

Sam Dolan

31,966
10
88
84

5

`out = out or {}` not only has a perlish whiff about it but also will fail if the caller passes in their own empty mapping object. `if out is None: out = {}` is preferable. – John Machin Oct 08 '10 at 02:00

score 3 · Answer 3 · answered Oct 08 '10 at 01:34

You are modifying the value of the function keyword parameter out in your method.

This blog post explains it succintly:

expressions in default arguments are calculated when the function is defined, not when it’s called.

The function is defined when the class is created, not for each instance. If you modify it like so, the problem goes away:

def test(self,out=None):
        if out is None:
                out = {}
        key = "key"
        for i in ['a','b','c','d']:
                if key in out:
                        out[key] += ','+i
                else:   
                        out[key] = i 
        return out

Why does initializing a variable via a python default variable keep state across object instantiation?

3 Answers3

Linked