Python references and loops

Question

I'm used to perl, where references and variables are explicitly separated. Starting to use python makes life very confusing. I want to work in a loop, creating objects and adding them to a list, but the results are frustrating. I have read that functions define variable scope, but even encapsulating my loop code within functions doesn't seem to cut it.

An example is as follows:

mylist = []

class thing:
    def __init__(self, stuff=[]):
        self.stuff = stuff

for i in range(10):
    obj = thing()
    obj.stuff.append(1)
    mylist.append(obj)

len(mylist[0].stuff)

I would expect the length of the "stuff" list of each object to be 1, but it comes out as 10. I know I can use deep copy to explicitly separate the newly created objects from what's gone before, but is there a more beautiful way?

Does this answer your question? ["Least Astonishment" and the Mutable Default Argument](https://stackoverflow.com/questions/1132941/least-astonishment-and-the-mutable-default-argument) — jasonharper, Jul 09 '20 at 03:50

User 10482 · Accepted Answer · 2020-07-09T03:23:21.137

The issue here is that you have the same list object being used as the default for all the objects you created with thing()

class thing:
    def __init__(self, stuff=[]):
        self.stuff = stuff

for i in range(3):
    obj = thing()
    print(id(obj.stuff))

# Output
140710582002368
140710582002368
140710582002368

One possible solution to this would be not to use the default, which would look like this

class thing:
  def __init__(self, stuff):
    self.stuff = stuff

for i in range(10):
  obj = thing(stuff = list())
  print(id(obj.stuff))

# Output
139911079590016
139911079590144
139911079590208

This assigns a new list every iteration of the for loop to the stuff member.

score 0 · Answer 2 · answered Jul 09 '20 at 03:28

Trying my hand at this, When the mutable list object is defined as a parameter it is passed around to the classes.


class thing:
    def __init__(self, stuff=[]):
        self.stuff = stuff


for i in range(10):
    obj = thing()
    print(obj)
    obj.stuff.append(1)
    print(obj.stuff)
    mylist.append(obj)
    print(mylist)

<main.thing object at 0x10ff4b5f8> [1] [<main.thing object at 0x10ff4b5f8>]

<main.thing object at 0x10ff4b978> [1, 1] [<main.thing object at 0x10ff4b5f8>, <main.thing object at 0x10ff4b978>]

<main.thing object at 0x10ff4b4e0> [1, 1, 1] [<main.thing object at 0x10ff4b5f8>, <main.thing object at 0x10ff4b978>, <main.thing object at 0x10ff4b4e0>]

<main.thing object at 0x10ff4b4a8> [1, 1, 1, 1] [<main.thing object at 0x10ff4b5f8>, <main.thing object at 0x10ff4b978>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b4a8>]

<main.thing object at 0x10ff4b5c0> [1, 1, 1, 1, 1] [<main.thing object at 0x10ff4b5f8>, <main.thing object at 0x10ff4b978>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b4a8>, <main.thing object at 0x10ff4b5c0>]

<main.thing object at 0x10ff4b588> [1, 1, 1, 1, 1, 1] [<main.thing object at 0x10ff4b5f8>, <main.thing object at 0x10ff4b978>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b4a8>, <main.thing object at 0x10ff4b5c0>, <main.thing object at 0x10ff4b588>]

<main.thing object at 0x10ff4b6d8> [1, 1, 1, 1, 1, 1, 1] [<main.thing object at 0x10ff4b5f8>, <main.thing object at 0x10ff4b978>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b4a8>, <main.thing object at 0x10ff4b5c0>, <main.thing object at 0x10ff4b588>, <main.thing object at 0x10ff4b6d8>]

<main.thing object at 0x10ff4b710> [1, 1, 1, 1, 1, 1, 1, 1] [<main.thing object at 0x10ff4b5f8>, <main.thing object at 0x10ff4b978>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b4a8>, <main.thing object at 0x10ff4b5c0>, <main.thing object at 0x10ff4b588>, <main.thing object at 0x10ff4b6d8>, <main.thing object at 0x10ff4b710>]

<main.thing object at 0x10ff4b668> [1, 1, 1, 1, 1, 1, 1, 1, 1] [<main.thing object at 0x10ff4b5f8>, <main.thing object at 0x10ff4b978>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b4a8>, <main.thing object at 0x10ff4b5c0>, <main.thing object at 0x10ff4b588>, <main.thing object at 0x10ff4b6d8>, <main.thing object at 0x10ff4b710>, <main.thing object at 0x10ff4b668>]

<main.thing object at 0x10ff4b748> [1, 1, 1, 1, 1, 1, 1, 1, 1, 1] [<main.thing object at 0x10ff4b5f8>, <main.thing object at 0x10ff4b978>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b4a8>, <main.thing object at 0x10ff4b5c0>, <main.thing object at 0x10ff4b588>, <main.thing object at 0x10ff4b6d8>, <main.thing object at 0x10ff4b710>, <main.thing object at 0x10ff4b668>, <main.thing object at 0x10ff4b748>]

When instanciating the list inside the function block it creates a new one each time.

Here is the output when using the class below.

class Thing:
    __ init__(self, stuff):
        if stuff:
            self.stuff = stuff
        else:
            self.stuff = []

for i in range(10):
     obj = thing()
    print(obj)
    obj.stuff.append(1)
    print(obj.stuff)
    mylist.append(obj)
    print(mylist)

Output

<main.thing object at 0x10ff4b518> [1] [<main.thing object at 0x10ff4b518>]

<main.thing object at 0x10ff4b4e0> [1] [<main.thing object at 0x10ff4b518>, <main.thing object at 0x10ff4b4e0>]

<main.thing object at 0x10ff4b6d8> [1] [<main.thing object at 0x10ff4b518>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b6d8>]

<main.thing object at 0x10ff4b550> [1] [<main.thing object at 0x10ff4b518>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b6d8>, <main.thing object at 0x10ff4b550>]

<main.thing object at 0x10ff4b7b8> [1] [<main.thing object at 0x10ff4b518>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b6d8>, <main.thing object at 0x10ff4b550>, <main.thing object at 0x10ff4b7b8>]

<main.thing object at 0x10ff4b7f0> [1] [<main.thing object at 0x10ff4b518>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b6d8>, <main.thing object at 0x10ff4b550>, <main.thing object at 0x10ff4b7b8>, <main.thing object at 0x10ff4b7f0>]

<main.thing object at 0x10ff4b668> [1] [<main.thing object at 0x10ff4b518>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b6d8>, <main.thing object at 0x10ff4b550>, <main.thing object at 0x10ff4b7b8>, <main.thing object at 0x10ff4b7f0>, <main.thing object at 0x10ff4b668>]

<main.thing object at 0x10ff4b6a0> [1] [<main.thing object at 0x10ff4b518>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b6d8>, <main.thing object at 0x10ff4b550>, <main.thing object at 0x10ff4b7b8>, <main.thing object at 0x10ff4b7f0>, <main.thing object at 0x10ff4b668>, <main.thing object at 0x10ff4b6a0>]

<main.thing object at 0x10ff4b748> [1] [<main.thing object at 0x10ff4b518>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b6d8>, <main.thing object at 0x10ff4b550>, <main.thing object at 0x10ff4b7b8>, <main.thing object at 0x10ff4b7f0>, <main.thing object at 0x10ff4b668>, <main.thing object at 0x10ff4b6a0>, <main.thing object at 0x10ff4b748>]

<main.thing object at 0x10ff4b828> [1] [<main.thing object at 0x10ff4b518>, <main.thing object at 0x10ff4b4e0>, <main.thing object at 0x10ff4b6d8>, <main.thing object at 0x10ff4b550>, <main.thing object at 0x10ff4b7b8>, <main.thing object at 0x10ff4b7f0>, <main.thing object at 0x10ff4b668>, <main.thing object at 0x10ff4b6a0>, <main.thing object at 0x10ff4b748>, <main.thing object at 0x10ff4b828>]

score 0 · Answer 3 · answered Jul 09 '20 at 03:37

This is a quirk of Python. The default value for an argument is only constructed once, at the time the function is defined. So, you can get this weird behavior:

def f(x=[]):
    x.append('x')
    return x

f()
# ['x']
f()
# ['x', 'x']
f()
# ['x', 'x', 'x']

It won't help to replace the [] with list(), because it will still be evaluated only once, when the function is defined. Then that same object will get reused as the default each time the function is called.

To get around this, you need to use a sentinel and then test explicitly for whether it's been overridden. This is a common pattern:

def f(x=None):
    if x is None:
        x = []
    x.append('x')
    return x

f()
# ['x']
f()
# ['x']
f(['a', 'b'])
# ['a', 'b', 'x']

Thank you all - I'm slowly getting used to the way this snake operates — Anon Ymouse, Jul 15 '20 at 05:08

Python references and loops

3 Answers3