Why the behavior is different between Python2 and Python3

Question

In Python2, it's valid:

#!/usr/bin/python

class ListNode(object):
    def __init__(self, val=0, next=None):
        self.val = val
        self.next = next

a = ListNode(0)
b = ListNode(1)

print(a < b)

Output: True

But same code in Python3, it will raise exception:

#!/usr/bin/python3

class ListNode(object):
    def __init__(self, val=0, next=None):
        self.val = val
        self.next = next

a = ListNode(0)
b = ListNode(1)

print(a < b)

Raise exception:

Traceback (most recent call last):
  File "c.py", line 11, in <module>
    print(a < b)
TypeError: '<' not supported between instances of 'ListNode' and 'ListNode'

Why it's different?

add:

I can add __lt__ method to ListNode to avoid the exception: ListNode.__lt__ = lambda a, b: id(a) - id(b).

But why there's no need for Python2 to add the __lt__ method?

In Python-3.x, you must [implement](https://stackoverflow.com/questions/3588776/how-is-eq-handled-in-python-and-in-what-order) the method `__eq__` to compare objects. — DYZ, Aug 28 '20 at 05:08
@DYZ: Well, a default `__eq__` is provided, it just compares by identity. `__lt__` doesn't have a useful default, so you have to write it yourself. — ShadowRanger, Aug 28 '20 at 05:13

score 5 · Accepted Answer · answered Aug 28 '20 at 05:12

5

In Python 2, when you lack __lt__ (or the older deprecated __cmp__), the default comparison rules kick in, which, in this case, eventually ends up comparing the memory addresses of the objects in question (before that, it's putting numbers before other stuff, and comparing everything else based on the string name of the class).

But this is almost never useful; if you haven't defined how to order instances of a class, arbitrarily ordering them by memory address (which changes each run) is silently misbehaving in 99% of code.

Python 3 fails loudly when that happens so people don't rely on sorting unsortable things by accident; if someone sorts [1, 'a', (), None] or even just a list of your instances with undefined ordering, raising an exception is just more helpful. If memory address ordering is really what's needed, you can always implement it like you did, but that's a vanishingly rare use case.

answered Aug 28 '20 at 05:12

ShadowRanger

143,180
12
188
271

Emm, it sounds reasonable. I understand why I have to implement methods such as `__lt__`, `__le__` in Python3 now. – Lane Aug 28 '20 at 05:17
@Lane: Yar. You don't need to implement all of them yourself though. [`functools.total_ordering`](https://docs.python.org/3/library/functools.html#functools.total_ordering) can be used as a decorator to turn just `__eq__` and `__lt__` into the full suite of rich comparison methods. Or you can write a [newfangled `dataclass`](https://docs.python.org/3/library/dataclasses.html) and just tell it `order=True` so it makes the comparison methods for you. – ShadowRanger Aug 28 '20 at 05:19
You mean by id. Are you sure that this is always a memory address? In python2, if I create an empty list, and then in a loop I create alternating integers and strings and append them to the list, and then sort it, all the integers end up at the start before all the strings. In CPython 2, the ids for the ints are much smaller numbers than the ids for the strings. (Obviously when comparing two ints in the list, or two strings, it will use the actual value -- but I'm talking here about when it compares ints with strings.) – alani Aug 28 '20 at 05:29
@alani: As I mentioned in the parenthetical, the default comparison rules only resort to comparing memory addresses after sorting things by their defined comparators when available, and the names of their types when they're not defined. It was an ugly, ugly mess, with lots of subtleties; it's just in the OP's case all the types are the same, so it ends up not mattering, and only the memory addresses count. CPython's `id` is the memory address (not necessarily the case in other interpreters, but the fallback sort wasn't a language defined behavior IIRC, just a CPython implementation detail). – ShadowRanger Aug 28 '20 at 05:35
@alani: Note: Even the "names of their types" thing is glossing over details (old-style and new-style classes wouldn't sort together, `None` sorts before everything else, numbers *do* sort together, etc.). Suffice to say, Python 2's fallback comparison was incredibly complicated, and died a well-deserved death. I'd like to let it rest in peace. :-) – ShadowRanger Aug 28 '20 at 05:39
I understand that it is only a fallback, but when comparing an int with a string, the int might have an id like 16752912 and the string might have a much bigger id like 139841655459008. I don't know much about memory management, but I am guessing that it means that they are using entirely different schemes for assigining ids, rather than that they are at widely different memory addresses. Is this correct? Certainly the integers always seem to be treated as less than the strings regardless of when the were created, consistent with sorting by ID. – alani Aug 28 '20 at 05:39
@alani: They're at wildly different memory addresses. [CPython's `id` is (as an implementation detail) the memory address](https://docs.python.org/3/library/functions.html#id), per the docs: "CPython implementation detail: This is the address of the object in memory." CPython has a complicated small object allocator that will confuse things, and it uses free lists, a small `int` cache, and `str` literal interning that messes with allocation ordering. All of this means that small (semi-)fixed size stuff like `int` gets put in different allocator "buckets" from larger, variable length `str`. – ShadowRanger Aug 28 '20 at 05:45

Why the behavior is different between Python2 and Python3

1 Answers1