-1

In a Python 3.8.5 shell I have :

>>> id([])
140370451791744
>>> id([])
140370451791744
>>> id([])

And in IPython 7.18.1 running Python 3.8.5 I have :

In [1]: id([])
Out[1]: 139870334633536

In [2]: id([])
Out[2]: 139870334633152

This leads me to the following question : why IPython behavior is different than python's one ?

This is related to a previous question I had Which interpreter is used by ipython? showing that both IPython and Python interactive shells use the same python implementation which is CPython.

To me, IPython is only a shell thing and should just forward interpretation of python's code to the underlying interpreter which is the same as the Python's one. Hence, the result of evaluating the same code should be the same (not the address of the object of course, but the fact of reusing the same one)

Note: I can easily imagine some reasons explaining why sometimes the interpreter is able to reuse the same memory location for a new object and sometimes not but I can't explain why the behaviors differs here since both use the same implementation.

Is this a duplicate: to people asking if this is a duplicate of How unique is Python's id()? the answer is clearly no and I tried to do my best to explain why above. Maybe it's not clear ?

Manuel Selva
  • 18,554
  • 22
  • 89
  • 134
  • Does this answer your question? [How unique is Python's id()?](https://stackoverflow.com/questions/52096582/how-unique-is-pythons-id) – buran Nov 03 '20 at 20:08
  • 5
    Whether two different objects, with disjoint lifetimes, have the same id is *COMPLETELY IRRELEVANT FOR ALL PURPOSES*. There is absolutely no situation where it is guaranteed to happen, there is absolutely no situation where it is guaranteed *not* to happen. IPython happens to execute more Python code in this case than plain CPython does (generating those fancy In/Out prompts, for example), so all that can be said is that these are two different situations. – jasonharper Nov 03 '20 at 20:09
  • @buran unfortunately no. I (tried to at least) explained that my question is about the different behavior in the two contexts. – Manuel Selva Nov 03 '20 at 20:11
  • @jasonharper are you saying that IPython is written in Python ? (and I definitely agree with what you said on ids, and it's not the purpose of my question) – Manuel Selva Nov 03 '20 at 20:12
  • You misunderstand the purpose of `id()` if you think this is a question that can be answered sensibly. – Mark Ransom Nov 03 '20 at 20:28
  • @MarkRansom I don't think so. But thanks to the comment of jasonharper I think I got my answer. I thought that IPython was written in C and calling the interpreter and hence, that in the two contexts the Python code executed was exactly the same. It is not the case, that's why in IPython the second [] instance is not allocated at the same place. Python code is happening in between. – Manuel Selva Nov 03 '20 at 20:36
  • 1
    Yes, IPython is written in Python. *Pure* Python, even, or at least the version I just checked is. – John Bollinger Nov 03 '20 at 20:37
  • And still, I am wondering if my question was not clear ? And why it has been down-voted. To me the question is valid : why behavior is not the same ? Answer : because Ipython is written in Python and then code happens in between the two empty lists creation, while it is not the case in the python interpreter. – Manuel Selva Nov 03 '20 at 20:38
  • 2
    downvoting might better better explained in psychology forum. But in my own words, on average, people do not like people who ask inconvenient questions. – Serge Nov 03 '20 at 21:29
  • 2
    `ipython` maintains a longer history of outputs, which may affect the details of memory use and reuse. The surest way of generating empty lists with distinct ids is to collect them in a list, eg. `alist = [[],[],[]]`. Now look at `[id(i) for i in alist]`. In general there isn't much value in looking at the `id()`. – hpaulj Nov 03 '20 at 22:02

3 Answers3

0

Try to run id([]) twice through one source file in IPython. It should give you the same results as with regular Python. Also IPython will not result in any errors due tot he behavior above. You can check the locals() and globals() after each command to troubleshoot further or understand more. It seems IPython garbage collects the first one immediately.

Aaj Kaal
  • 1,205
  • 1
  • 9
  • 8
0

That is an interesting find. Interactive shell's id indeed appear somewhat more stable. Yet if you tinker a little bit more in the shell you can observe different empty list's id even in the same session of the interactive Python shell.

>>> id([])
140490599993424
>>> [] is []
False
>>> id([])
140490599925376
>>> id([])
140490599925376
>>> [] is []
False
>>> id([])
140490599993424

Lesser id stability in iPython might be related to to garbage collection but better ask iPython developers, either way it is obvious that iPython is more complex that shell.

Identity is a bit a loose concept, do not rely on it too much

>>> 123456789 is 123456789
True
>>> 12345678923456789 is 123456789123456789
False
>>> frozenset({}) is frozenset({})
True
>>> tuple() is tuple()
True
>>> list() is list()
False
>>> set() is set()
False
>>> id(12345678923456789)
139811567040912
>>> id(12345678923456789)
139811567040912
>>> id(123456789234567893)
139811567040912

find more at Two variables in Python have same id, but not lists or tuples

Edited: I removed the paragraph about the role non-determinism in computation which was attracting off-topic discussions.

Serge
  • 3,387
  • 3
  • 16
  • 34
  • Modern computers are deterministic by definition. Any imperfections such as defects or lack of care should not (and almost never do) introduce discrepancies such as different ID. This is achieved through numerous safeguards. Quick note -- I am not talking about code bugs, just the non-determinism in consumer electronics in general. – RafazZ Nov 03 '20 at 22:30
  • Who told you that lie about modern (or ancient) computers? Why id's should be same? It's fine to have volatile id as long as behavior is well documented – Serge Nov 03 '20 at 22:39
  • It is not about the volatility of the id, but the reasoning behind it. The reason the id's are different is documented, and it is NOT the non-determinism of the computers. It is all in the implementation of the memory management in the context of the multi-threaded execution. About the "lie" -- the modern computers are deterministic for all consumer applications. If it looks non-deterministic, it is explainable and predictable, assuming you have the knowledge about the current state and the implementation. We can move this conversation to a separate chat, if you want to continue the discussion – RafazZ Nov 03 '20 at 22:48
  • The python shell runs one thread. Yet, yes, multi-threading and concurrency are great source of non-determinism. – Serge Nov 03 '20 at 23:00
  • I think there is something lost in the definitions here: if you say that the determinism also includes the knowledge of an immediate user, then literally everything in the electronics is non-deterministic -- any memory allocation becomes non-deterministic. However, in the CS the determinism is clearly defined as "given a specific initial state, the final state is always the same". That is why there is no true random number generator and that is why we can compute pi to millions of digits. And only VERY unlucky hardware defects (as you mentioned) can affect this behavior. – RafazZ Nov 03 '20 at 23:28
  • There are true number generators, well, according to science. They however based on external processes, believed to be random, such as quantum effect. For practical needs something like noise of the processor fan or temperature can be considered random. – Serge Nov 03 '20 at 23:51
  • I think you are a bit lost in attempt to define determinism. I even did not mentioned immediate user. Contrary to you I rather think the more knowledge of system or user you have the more predictable and deterministic results are. Yet lack of knowledge, or parameters which are out of our control can be interpreted as nondeterminism. Say if my software creates an object I do not know which place in memory it will be saved ahead each time. – Serge Nov 03 '20 at 23:51
  • https://en.wikipedia.org/wiki/Nondeterministic_programming https://en.wikipedia.org/wiki/Relativistic_programming – Serge Nov 04 '20 at 00:05
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/224120/discussion-between-rafazz-and-serge). – RafazZ Nov 04 '20 at 20:35
0

This is a very interesting observation, and not a lot of people are aware that IPython uses two threads when executing the statements. One thread is used for history, and another for new execution. Whenever you are calling one id after another, the second one is executed in a different thread, which makes it have a different id.

You can play with your finding:

in IPython:

import threading
print(threading.activeCount())  # Should print "2"

# The statetement below should be printing alternating id numbers
# Such as "1 2 1 2" or smthng like that
print(id([])); print(id([])); print(id([])); print(id([]));

However, plain old python shell uses a single thread, which means that id's of the same objects will be the same at all times.

in Python shell

import threading
print(threading.activeCount())  # Should print "1"

# The statetement below should be printing the same values
# Such as "1 1 1 1" or smthng like that
print(id([])); print(id([])); print(id([])); print(id([]));

Some more info:

RafazZ
  • 4,049
  • 2
  • 20
  • 39
  • You missed a few bracket, yet I just tried with ipython and see a new id each time, not two alternating ids ```In [1]: print(id([])); print(id([])); print(id([])) 139808971501632 139808971823744 139808971503296``` new id each time – Serge Nov 04 '20 at 01:05
  • Thanks -- fixed the brackets. For the IDs being different, check the number of threads that your machine is running – RafazZ Nov 04 '20 at 20:46
  • Also, try printing > 3 statements -- usually if the computer is a little too fast, the threads join and release before the next statement is executed. Putting more statements forces the overlap before the release of the locks, and the ids should be duplicated in some of them – RafazZ Nov 04 '20 at 20:49