3

I was playing a bit in my python shell while learning about mutability of objects.

I found something strange:

>>> x=5.0
>>> id(x)
48840312
>>> id(5.0)
48840296
>>> x=x+3.0
>>> id(x) # why did x (now 8.0) keep the same id as 5.0?
48840296
>>> id(5.0)
36582128
>>> id(5.0)
48840344

Why is the id of 5.0 reused after the statement x=x+3.0?

dimo414
  • 47,227
  • 18
  • 148
  • 244
darxtrix
  • 2,032
  • 2
  • 23
  • 30
  • Also see, http://stackoverflow.com/questions/6101379/what-happens-behind-the-scenes-when-python-adds-small-ints – Abhinav Sarkar Dec 23 '13 at 09:37
  • And http://me.veekun.com/blog/2012/03/24/python-faq-equality/ (section 'is and built ins') – jonrsharpe Dec 23 '13 at 09:38
  • The question you referred can only answer the fact for the long integers but here my another question is that why x got the id of 5.0 after being changed? – darxtrix Dec 23 '13 at 09:42
  • Note this has nothing to do with mutability. You pose an interesting question about why the result of `id(5.0)` is changing, but the value, `5.0`, remains constant throughout your example. I suspect the close votes stem from this mis-categorization of the problem. – dimo414 Dec 23 '13 at 09:49
  • yes i know this but i put the question in the mutability category for the integer case,i mentioned above the floats' case. – darxtrix Dec 23 '13 at 09:56
  • Neither integers nor floats in Python are mutable. – dimo414 Dec 23 '13 at 09:59
  • Edited it as suggested. – darxtrix Dec 23 '13 at 10:16
  • Despite the similarity, I feel this is a very different question than the marked duplicate. The earlier question asks why a variable has different `id()`s over time, while this question asks why a variable, when changed, still has the same `id()`. – dimo414 Dec 23 '13 at 10:17
  • @Eric,@alko,@joaquin,i have changed the post.will you remove the duplicate. – darxtrix Dec 23 '13 at 10:26
  • I can not remove duplicate flag afaik, just vote to reopen. You could also write a new question addressing the point not covered by the refered answers, as the duplicate notice indicates. This is probably better than drastically edit the question as it could generate confusion. – joaquin Dec 23 '13 at 10:30
  • @dimo414 i am going to re ask this question as suggested by the people who marked it duplicate. – darxtrix Dec 23 '13 at 10:43
  • @Eric,will you vote to reopen this post.I have changed this. – darxtrix Dec 23 '13 at 11:03
  • @alko,will you vote to reopen this post.I have changed this. – darxtrix Dec 23 '13 at 11:03
  • @UmNyobe will you vote to reopen this post.I have changed this. – darxtrix Dec 23 '13 at 11:04

2 Answers2

2

Fundamentally, the answer to your question is "calling id() on numbers will give you unpredictable results". The reason for this is because unlike languages like Java, where primitives literally are their value in memory, "primitives" in Python are still objects, and no guarantee is provided that exactly the same object will be used every time, merely that a functionally equivalent one will be.

CPython caches the values of the integers from -5 to 256 for efficiency (ensuring that calls to id() will always be the same), since these are commonly used and can be effectively cached, however nothing about the language requires this to be the case, and other implementations may chose not to do so.

Whenever you write a double literal in Python, you're asking the interpreter to convert the string into a valid numerical object. If it can, Python will reuse existing objects, but if it cannot easily determine whether an object exits already, it will simply create a new one.

This is not to say that numbers in Python are mutable - they aren't. Any instance of a number, such as 5.0, in Python cannot be changed by the user after being created. However there's nothing wrong, as far as the interpreter is concerned, with constructing more than one instance of the same number.

Your specific example of the object representing x = 5.0 being reused for the value of x += 3.0 is an implementation detail. Under the covers, CPython may, if it sees fit, reuse numerical objects, both integers and floats, to avoid the costly activity of constructing a whole new object. I stress however, this is an implementation detail; it's entirely possible certain cases will not display this behavior, and CPython could at any time change its number-handling logic to no longer behave this way. You should avoid writing any code that relies on this quirk.

The alternative, as eryksun points out, is simply that you stumbled on an object being garbage collected and replaced in the same location. From the user's perspective, there's no difference between the two cases, and this serves to stress that id() should not be used on "primitives".

dimo414
  • 47,227
  • 18
  • 148
  • 244
  • This answer currently addresses why taking `id(5.0)` repeatedly produced different results, but it's lacking an explanation for why the ID of the first `5.0` could be reused for `8.0`. – user2357112 Dec 23 '13 at 10:03
  • Great,grabbed some good things you pointed out.But as you said that every time you use a statement like x=5.0 will create a new object and x will refer to that and if now i change x by saying x=x+3.0; then x will set to refer a new object but here above why again referred the location of 5.0. – darxtrix Dec 23 '13 at 10:07
  • @black_perl, clarified my answer. I don't believe I said Python would *always* create a new object however, merely that it often will. The exact cases where it does or doesn't create a new object are (generally) hidden from the user, and shouldn't be relied on. – dimo414 Dec 23 '13 at 10:13
  • It's not the object for `x = 5.0` that was reused, but the `float` created for `id(5.0)`. When the latter statement returned that `float` was put on the `float` freelist to be reused by `x+3.0`. An in-place add can't reuse the same float object (at least with current implementation); the original's reference count isn't decremented until after the new `float` is created. – Eryk Sun Dec 23 '13 at 10:17
  • @eryksun I'm willing to defer to someone with more knowledge of the CPython internals, but as I interpret the documentation of the [`id()` function](http://docs.python.org/2/library/functions.html#id), `id()` will only return the same value if the same object is being used (unless the memory address is recycled, which seems like a stretch). Therefore at a glance it appears CPython is internally updating the object `x` is referencing from a value of `5.0` to a value of `8.0`, without actually replacing the object. – dimo414 Dec 23 '13 at 10:23
  • The memory is recycled. Allocating memory isn't cheap, so many types use a freelist of unreferenced objects. There's also a small object allocator that allocates 256 KiB arenas that are split into 4 KiB pools. A pool has a fixed storage size, anywhere from 8 to 512 bytes (256 prior to 3.3) in steps of 8 (larger than 512 bytes defers to `malloc` and friends), so it's common to see address reuse here as well. – Eryk Sun Dec 23 '13 at 10:34
  • Thanks @eryksun, I didn't expect Python to be quite that predictable with it's memory allocation, but it's certainly reasonable to imagine. Updated my answer. – dimo414 Dec 23 '13 at 11:04
  • If you have some free time to kill, take a look at [`PyFloat_FromDouble`](http://hg.python.org/cpython/file/c3896275c0f6/Objects/floatobject.c#l114). Note how it reuses the `ob_type` slot to implement a linked list of unreferenced `float` objects. But even if `free_list` is empty, note also that it allocates a new object with `PyObject_MALLOC`. This is the small-object allocator, implemented in [obmalloc.c](http://hg.python.org/cpython/file/c3896275c0f6/Objects/obmalloc.c), so even at this point it could reuse an address from a pool. – Eryk Sun Dec 23 '13 at 11:07
  • Well,tried this:'>>> a=3.0 >>> id(a) 48840264 >>> id(3.0) 48840312 >>> a=3.0+8.0 >>> id(a) 48840232' and this time python doesn"t used the id of 3.0 ,means it was a matter of chance that python used the garbage collected values.Thanks to you and eryksun – darxtrix Dec 23 '13 at 11:15
  • @dimo414, just give a try and comment. – darxtrix Dec 23 '13 at 11:22
0

The Devil is in the details

PyObject* PyInt_FromLong(long ival)

Return value: New reference.
Create a new integer object with a value of ival.

The current implementation keeps an array of integer objects for all integers between -5 and 256, when you create an int in that range you actually just get back a reference to the existing object. So it should be possible to change the value of 1. I suspect the behaviour of Python in this case is undefined. :-)

Note This is true only for CPython and may not apply for other Python Distribution.

Abhijit
  • 62,056
  • 18
  • 131
  • 204
  • This is unhelpful and misleading. The low-level `PyInt_FromLong` C function has little to do with the OP's problem, and the documentation quote is likely to cause confusion about whether ints are mutable. – user2357112 Dec 23 '13 at 09:54
  • @user2357112: Its more than immutability of integer. Its about caching a certain range of integers for faster processing. This is an official reference from the Doc which claims it does and conforms to OPs observation. So if you are the downvoter, you should seriously consider if you understand the problem and the reason behind it. – Abhijit Dec 23 '13 at 09:56
  • This answer also does nothing to address the confusion about Python's handling of floats. It does not explain why the ID of `5.0` was reused for a different number or why taking the ID of `5.0` repeatedly gave different results. A good answer to this question must address those points. – user2357112 Dec 23 '13 at 10:00