7

I am confused with Python referencing. Consider the following example:

My task : To edit each element in the list

d = { 'm': [1,2,3] }
m = d['m']
m = m[1:]   # m changes its reference to the new sliced list, edits m but not d (I wanted to change d)

Similarly:

d = { 'm': [1,2,3] }
m = d['m']
m = m[0]    # As per python referencing, m should be pointing to d['m'] and should have edited d

In python everything goes by reference, then when is a new object created? Do we always need copy and deepcopy from copy module to make object copies?

Please clarify.

mechanical_meat
  • 163,903
  • 24
  • 228
  • 223
Yugal Jindle
  • 44,057
  • 43
  • 129
  • 197

3 Answers3

14

In Python a variable is not a box that holds things, it is a name that points to an object. In your code:

  • d = { 'm': [1,2,3] } --> binds the name d to a dictionary
  • m = d['m'] --> binds the name m to a list
  • m = m[1:] --> binds the name m to another list

Your third line is not changing m itself, but what m is pointing to.

To edit the elements in the list what you can do is:

m = d['m']
for i, item in enumerate(m):
    result = do_something_with(item)
    m[i] = result
Ethan Furman
  • 63,992
  • 20
  • 159
  • 237
  • Is there any other way to edit each element of this list apart from `enumerate` ? – Yugal Jindle Mar 15 '12 at 17:47
  • "A name that points to an object" might be confusing, since "pointer" isn't Python terminology. I'd just say "it is a name for an object". – Karl Knechtel Mar 15 '12 at 18:13
  • "Your third line is not changing m itself, but what m is pointing to" could be a little unclear for the reader. To be more explicit: "the third line changes m from pointing to one thing (d['m']) to pointing to another thing ([2,3])." – Chris Morris Mar 15 '12 at 18:18
  • @Ethan Furman "Your third line is not changing m itself, but what m is pointing to." It's my understanding that python strings are considered "immutable", does that mean, instead of changing the string at the memory address, the name will have to bind to a different memory address with a different name? – ScottyBlades Apr 21 '20 at 23:47
  • @ScottyBlades: Mutable/immutable doesn't matter, and memory addresses are not typically a concern for Python programmers. In `a = b`, `a` has been bound to (or is another name for) the same object that `b` refers to. `a` is not a separate copy; if `b` is mutable and mutated, than `a` will also show the change as `a` and `b` are simply two names for the same thing; likewise, if you did `del b` you have not removed the object that `b` refers to, only removed the name `b` -- you can still access the object via it's `a` name. – Ethan Furman Apr 22 '20 at 00:19
  • `first = 30 second = first first = 20 print("first: ", first, "second: ", second)` – ScottyBlades Apr 22 '20 at 01:10
  • @ScottyBlades: Yes, you assigned the name `first` to the object `30`, then assigned `second` to the same object, then reassigned `first` to a different object (which has zero impact on the object that now only `second` is assigned to), then verified that with a `print()`. The only way for `second` to see the change would be for the `first` object to be mutable, and then to mutate it: `first = [20]` `second = first` `first[0] = 30 # mutate the list first is assigned to` `print("first: ", first, "second: ", second)` – Ethan Furman Apr 22 '20 at 06:41
5

Ethan Furman did an excellent job of explaining how Python internals work, I won't repeat it.

Since m really does represent the list inside the dictionary, you can modify it. You just can't reassign it to something new, which is what happens when you use = to equate it to a new slice.

To slice off the first element of the list for example:

>>> m[0:1] = []
>>> d
{'m': [2, 3]}
Community
  • 1
  • 1
Mark Ransom
  • 299,747
  • 42
  • 398
  • 622
  • 4
    I hate drive by downvotes, if I did something wrong I like to find out why. This answer has been tested in Python 2.7 using a recommended idiom: http://docs.python.org/library/stdtypes.html#mutable-sequence-types – Mark Ransom Mar 15 '12 at 17:56
  • 3
    To delete items or slices from a list I usually prefer the more explicit `del m[0]`. Of course this is a stylistic preference, and I've got no idea why someone downvoted. My guess is that many of those seemingly random downvotes result from the too small displays of smartphones. – Sven Marnach Mar 15 '12 at 21:01
4

In python everything goes by reference

In Python, everything is a reference, and the references get passed around by value.

If you want to use those terms. But those terms make things harder to understand.

Much simpler: in Python, a variable is a name for an object. = is used to change what object a name refers to. The left-hand side can refer to part of an existing object, in which case the whole object is changed by replacing that part. This is because the object, in turn, doesn't really contain its parts, but instead contains more names, which can be caused to start referring to different things.

then when is a new object created ?

Objects are created when they are created (by using the class constructor, or in the case of built-in types that have a literal representation, by typing out a literal). I don't understand how this is relevant to the rest of your question.

m = m[1:]   # m changes its reference to the new sliced list

Yes, of course. Now m refers to the result of evaluating m[1:].

edits m but not d (I wanted to change d)

Yes, of course. Why would it change d? It wasn't some kind of magic, it was simply the result of evaluating d['m']. Exactly the same thing happens on both lines.

Let's look at a simpler example.

m = 1
m = 2

Does this cause 1 to become 2? No, of course not. Integers are immutable. But the same thing is happening: m is caused to name one thing, and then to name another thing.

Or, another way: if "references" were to work the way you expect, then the line m = m[1:] would be recursive. You're expecting it to mean "anywhere that you see m, treat it as if it meant m[1:]". Well, in that case, m[1:] would actually mean m[1:][1:], which would then mean m[1:][1:][1:], etc.

Karl Knechtel
  • 62,466
  • 11
  • 102
  • 153