10

According to the object.__eq__() documentation, the default (that is, in the object class) implementation for == is as follows:

True if x is y else NotImplemented

Still following the documentation for NotImplemented, I inferred that NotImplemented implies that the Python runtime will try the comparison the other way around. That is try y.__eq__(x) if x.__eq__(y) returns NotImplemented (in the case of the == operator).

Now, the following code prints False and True in python 3.9:

class A:
   pass
print(A() == A())
print(bool(NotImplemented))

So my question is the following: where does the documentation mention the special behavior of NotImplemented in the context of __eq__ ?

PS : I found an answer in CPython source code but I guess that this must/should be somewhere in the documentation.

jsbueno
  • 99,910
  • 10
  • 151
  • 209
Manuel Selva
  • 18,554
  • 22
  • 89
  • 134
  • 8
    The documentation is misleading and there's actually an issue tracking it:https://github.com/python/cpython/issues/83292 – whilrun May 30 '22 at 21:27
  • This answer https://stackoverflow.com/a/57134523/7517724 provides a link to the commit with GvRs explanation for the change, but that doesn't explain why it's not in the documentation. – Craig May 30 '22 at 21:27
  • I think it's interesting to note that `x = A()`, `y = A()`, `print(x==x)`, `print(x==y)` returns `True` and `False` respectively. – BeRT2me May 30 '22 at 21:30
  • 1
    @BeRT2me That's because `x == x` compares them using identity first: https://docs.python.org/3/c-api/object.html#c.PyObject_RichCompareBool – Ashwini Chaudhary May 30 '22 at 21:35
  • I thought this refers to the identity lookup in the memory register. But since A is not instantiated yet, it registers 2X inline? – Yaakov Bressler May 30 '22 at 21:36
  • 2
    @YaakovBressler They are both instantiated. I have no idea what you mean by "registers inline". In order for the objects in `A() == A()` to exist long enough to be compared, they have to be separate objects. The default logic will make them compare false, because they are not the same object. – Karl Knechtel May 30 '22 at 22:10
  • @Craig as I have now put in my answer, it *is* in the documentation - just not *this part*. – Karl Knechtel May 30 '22 at 22:11

1 Answers1

6

According to the object.__eq__() documentation, the default (that is, in the object class) implementation for == is as follows

No; that is the default implementation of __eq__. ==, being an operator, cannot be implemented in classes.

Python's implementation of operators is cooperative. There is hard-coded logic that uses the dunder methods to figure out what should happen, and possibly falls back on a default. This logic is outside of any class.

You can see another example with the built-in len: a class can return whatever it likes from its __len__ method, and you can in principle call it directly and get a value of any type. However, this does not properly implement the protocol, and len will complain when it doesn't get a non-negative integer back. There is not any class which contains that type-checking and value-checking logic. It is external.

Still following the documentation for NotImplemented, I inferred that NotImplemented implies that the Python runtime will try the comparison the other way around. That is try y.__eq__(x) if x.__eq__(y) returns NotImplemented (in the case of the == operator).

NotImplemented is just an object. It is not syntax. It does not have any special behavior, and in Python, simply returning a value does not trigger special behavior besides that the value is returned.

The external code for binary operators will try to look for the matching __op__, and try to look for the matching __rop__ if __op__ didn't work. At this point, NotImplemented is not an acceptable answer (it is a sentinel that exists specifically for this purpose, because None is an acceptable answer). In general, if the answer so far is still NotImplemented, then the external code will raise NotImplementedError.

As a special case, objects that don't provide their own comparison (i.e., the default from object is used for __eq__ or __ne__) will compare as "not equal" unless they are identical. The C implementation repeats the identity check (in case a class explicitly defines __eq__ or __ne__ to return NotImplemented directly, I guess). This is because it is considered sensible to give this result, and obnoxious to make == fail all the time when there is a sensible default.

However, the two objects are still not orderable without explicit logic, since there isn't a reasonable default. (You could compare the pointer values, but they're arbitrary and don't have anything to do with the Python logic that got you to that point; so ordering things that way isn't realistically useful for writing Python code.) So, for example, x < y will raise a TypeError if the comparison logic isn't provided. (It does this even if x is y; you could reasonably say that <= and >= should be true in this case, and < and > should be false, but it makes things too complicated and is not very useful.)

[Observation: print(bool(NotImplemented)) prints True]

Well, yes; NotImplemented is an object, so it's truthy by default; and it doesn't represent a numeric value, and isn't a container, so there's no reason for it to be falsy.

However, that also doesn't tell us anything useful. We don't care about the truthiness of NotImplemented here, and it isn't used that way in the Python implementation. It is just a sentinel value.

where does the documentation mention the special behavior of NotImplemented in the context of __eq__ ?

Nowhere, because it isn't a behavior of NotImplemented, as explained above.

Okay, but that leaves underlying question: where does the documentation explain what the == operator does by default?

Answer: because we are talking about an operator, and not about a method, it's not in the section about dunder methods. It's in section 6, which talks about expressions. Specifically, 6.10.1. Value comparisons:

The default behavior for equality comparison (== and !=) is based on the identity of the objects. Hence, equality comparison of instances with the same identity results in equality, and equality comparison of instances with different identities results in inequality. A motivation for this default behavior is the desire that all objects should be reflexive (i.e. x is y implies x == y).

Kelly Bundy
  • 23,480
  • 7
  • 29
  • 65
Karl Knechtel
  • 62,466
  • 11
  • 102
  • 153
  • The last part is where I looked as well and it didn't quite fully convince me, the "equality comparison of instances with different identities results in inequality" could just mean that `!=` results in `True`. But I guess you're right, it's supposed to also mean that `==` results in `False`. – Kelly Bundy May 30 '22 at 22:17
  • `(x == y) == not (x != y)` is a separate invariant, listed later in the section: "Inverse comparison should result in the boolean negation. In other words, the following expressions should have the same result:". – Karl Knechtel May 30 '22 at 22:40
  • That's under *"User-defined classes that customize their comparison behavior should"*, though, so it's talking about what *our* classes *should* do at the `__eq__` level. Not about what Python does do at the `==` level. – Kelly Bundy May 30 '22 at 22:46
  • I guess, but I feel like we're getting into serious nitpicking now. – Karl Knechtel May 30 '22 at 22:49
  • @KarlKnechtel Thank you very much for your detailed answer. I still believe that the documentation could be way more clear. We can read "`x==y calls x.__eq__(y)`" and "By default, object implements `__eq__()` by using is, returning `NotImplemented` in the case of a false comparison: `True if x is y else NotImplemented`." in the data model section of the documentation. Reading this, and not knowing that section 6 speaks also about `==`, I was lost. – Manuel Selva May 31 '22 at 12:49
  • There are many places where the documentation could be more clear. The dev team has recently been turning more attention to that sort of thing, which is part of the same effort as 3.10's improved error messages. – Karl Knechtel May 31 '22 at 14:37