Python: Why are int & list function parameters differently treated?

Question

We all know the dogma that global variables are bad. As I began to learn python I read parameters passed to functions are treated as local variables inside the funktion. This seems to be at least half of the truth:

def f(i):
    print("Calling f(i)...")
    print("id(i): {}\n".format(id(i)))
    print("Inside f(): i += 1")
    i += 1
    print("id(i): {}".format(id(i)))
    return

i = 1
print("\nBefore function call...")
print("id(i): {}\n".format(id(i)))
f(i)

This evaluates to:

Before function call...
id(i): 507107200

Calling f(i)...
id(i): 507107200

Inside f(): i += 1
id(i): 507107232

As I read now, the calling mechanism of functions in Python is "Call by object reference". This means an argument is initially passed by it's object reference, but if it is modified inside the function, a new object variable is created. This seems reasonable to me to avoid a design in which functions unintendedly modify global variables.

But what happens if we pass a list as an argument?

def g(l):
    print("Calling f(l)...")
    print("id(l): {}\n".format(id(l)))
    print("Inside f(): l[0] += 1")
    l[0] += 1
    print("id(l): {}".format(id(l)))
    return

l = [1, 2, 3]
print("\nBefore function call...")
print("id(l): {}\n".format(id(l)))
g(l)

This results in:

Before function call...
id(l): 120724616

Calling f(l)...
id(l): 120724616

Inside f(): l[0] += 1
id(l): 120724616

As we can see, the object reference remains the same! So we work on a global variable, don't we?

I know we can easily overcome this by passing a copy of the list to the function with:

g(l[:])

But my question is: What is the reason the implement two different behaviors of function parameters in Python? If we intend to manipulate a global variable, we could also use the "global"-keyword for list like we would do for integers, couldn't we? How is this behavior consistent with the zen of python "explicit is better than implicit"?

Answer 1

Python has two types of objects - mutable and inmutable. Most of build-in types, like int, string or float, are inmutable. This means they cannot change. Types like list, dict or array are mutable, which means that their state can be changed. Almost all user defined objects are mutable too.

When you do i += 1, you assign a new value to i, which is i + 1. This doesn't mutate i in any way, it just says that it should forget i and replace it with value of i + 1. Then i becomes replaced by a completely new object. But when you do i[0] += 1 in list, you say to the list that is should replace element 0 with i[0] + 1. This means that id(i[0]) will be changed with new object, and the state of list i will change, but it's identity remains the same - it's the same object it was, only changed.

Note that in Python this is not true for strings, as they are immutable and changing one element will copy the string with updated values and create new object.

Answer 2

Why are int & list function parameters differently treated?

They are not. All parameters are treated the same, regardless of type.

You are seeing different behavior between the two cases because you are doing different things to l.

First, let's simplify the += into an = and a +: l = l + 1 in the first case, and l[0] = l[0] + 1 in the second. (+= doesn't always equal an assignment and +; it depends on the runtime class of the object on the left side, which can override it; but here, for ints, it is equivalent to an assignment and +.) Also, the right side of the assignment just reads stuff and is not interesting, so let's just ignore it for now; so you have:

l = something (in the first case)
l[0] = something (in the second case)

The second one is "assigning to an element", which is actually syntactic sugar for a call to the method . __setitem__():

l.__setitem__(0, something)

So now you can see the difference between the two --

• In the first case, you are assigning to the variable l. Python is pass-by-value, so this has no effect on outside code. Assigning to the variable simply makes it point to a new object; it has no effect on the object that it used to point to. If you had assigned something to l in the second case, it would also have had no effect on the original object.
• In the second case, you are calling a method on the object pointed to by l. This method happens to be a mutating method on lists, and so modifies the contents of the list object, the original list object a pointer to which was passed in to the method. It is true that int (the runtime class of l in the first case) happens to have no methods that are mutating, but that is besides the point.

If you had done the same thing to l in both cases (if that were possible), then you can expect the same semantics.

Answer 3

This is pretty common across a bunch of languages (Ruby, for example).

The variable itself is scoped to the function. But that variable is just a pointer to an object floating around in memory somewhere -- and that object can be changed.

Answer 4

In Python everything is an object, and hence everything is represented by reference. The most notable thing about variables in Python is that they contain references to objects, not the objects themselves. Now, when arguments are passed to functions, they are passed by reference. Consequently, Inside the scope of a function, every parameter is assigned to the reference of the argument and then treated as a local variable inside the function. When you assign a new value to a parameter, you are changing the object it refers to, and so you have a new object and any changes to it (even if it's a mutable object) will not be seen outside the scope of the function in question, and not related anyway to the passed argument. That said, when you don't assign a new reference to the parameter, it stays holding the reference of the argument, and any changes to it (if and only if it's mutable) will be seen outside the scope of the function.