Why does my function overwrite a list passed as a parameter?

Question

I have created a function that takes a list as a parameter. It shuffles the list, replaces the first element and returns the new list.

import random
firstList=["a","b","c","d","e","f","g","h","i"]

def substitution(importedList):
    random.shuffle(importedList)
    importedList[0]="WORD"
    return importedList

The shuffle has no impact on my question. However, I was surprised to see that the returned importedList overwrites the original firstList.

>>> firstList
['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i']

>>> substitution(firstList)
['WORD', 'a', 'b', 'd', 'i', 'c', 'g', 'e', 'h']

>>> firstList
['WORD', 'a', 'b', 'd', 'i', 'c', 'g', 'e', 'h']

I have found a workaround by copying the list within the function, but it seems inefficient.

import random
firstList=["a","b","c","d","e","f","g","h","i"]
string="a"

def substitutionandcopy(importedList):
    copiedList=importedList[:]
    random.shuffle(copiedList)
    copiedList[0]="WORD"
    return copiedList

My question is why does the function replace the firstList? This would not happen if it were a string for example.

string="a"

def substituteString(foo):
    foo='b'
    return foo

---

>>> string
'a'

>>> substituteString(string)
'b'

>>> string
'a'

Answer 1

Strings, Ints, Tuples are immutable python types, so when you perform operations that change one of these types the new corresponding object is effectively created in memory each time. (Or you get an error if trying to change those in-place.)

Lists and dictionaries are mutable python types, so when you perform operations that change one of these types, the object stays the same, but it's parts (i.e., list elements) get changed.

So when you want to change a list, but want to leave the original intact you have to copy it yourself. Important thing, that there're two types of copying - shallow copy and deep copy.

Shallow copy can be done like so:

list_b = list_a[:] #using slice syntax

#or

list_b = list(list_a) #instantiating a new list from iterating over the old one

#or

import copy
list_b = copy.copy(list_a) #using copy module

Deep copy is done in the following way:

import copy
list_b = copy.deepcopy(list_a)

The difference between deep copy and shallow copy is...

When doing shallow copy, if mutable object contains other mutable objects, only the top one is copied. I.e. if a list contains other list, if top list is copied and then the inner list is changed in the copy, effectively the inner list will be changed both in the copy and in the original, because it's the same object in memory that is referenced in two different lists. Basicly shallow copy creates a new object with the same references stored in original object.

When doing deep copy, if mutable object contains other mutable objects, then inner mutable objects are copied too. I.e. as in previous example, if you change inner list in the copy, it changes only in the copy and the original is not affected. So deep copy copies everything, creates new structure in memory for everything in the object being copied, and not just references.

Answer 2

It does not replace the first list. The first list is passed by reference meaning that any mutations you perform on the list that is passed as the parameter, will also be performed on the list outside of the function, because it is the same list.

However, Strings and other basic types are not passed by reference and therefore any changes you make in your function scope is tot he local copy of the variable only.

Answer 3

As you found out, random.shuffle mutates the list in place:

random.shuffle(x[, random])

Shuffle the sequence x in place. The optional argument random is a 0-argument function returning a random float in [0.0, 1.0); by default, this is the function random().

Note that for even rather small len(x), the total number of permutations of x is larger than the period of most random number generators; this implies that most permutations of a long sequence can never be generated.

Strings are immutable in Python, all string operations return a new string instead. This is the "string" example from your question:

string="a"

def substitute_string(foo):
    foo = 'b'
    return foo

It is not really akin to the code from the substitution list in the first code block of the question. The equivalent code using a list would be this:

alist = [1, 2, 3]

def substitute_list(foo):
    foo = [4, 5, 6]
    return foo

And it works identically:

>>> alist
[1, 2, 3]

>>> substitute_list(alist)
[4, 5, 6]

>>> alist
[1, 2, 3]

Back to your solution, it could be:

def substitution_and_copy(imported_list):
    imported_list = imported_list[:]
    random.shuffle(imported_list)
    imported_list[0]="WORD"
    return imported_list

And no, assigning a new value to the argument will not mutate the original list, the same way you don't mutate the original string when you assign a new value to foo (also changed camelCase to snake_case, I'm a little nazy about PEP8).

[update]

What you have now, however, is what he already tried. "I have found a workaround by copying the list within the function, but it seems inefficient"

A list copy is not as inefficient as you may think, but this is not the point: as someone else pointed out, either you mutate the list in place and return nothing or return a new list - you can't have your cake and eat it.

Answer 4

From the docs on random.shuffle(): shuffle list x in place; return None. If you don't want that, you can use random.sample():

def substitutionandcopy(importedList):
    shuffledList = random.sample(importedList, len(importedList))
    shuffledList[0]="WORD"
    return shuffledList