2

In Python 3, when defining a subclass, why do you need to use cls as the first argument of __new__, but not to use self as the first argument of __init__?

An example:

class MyClass(object):
    def __new__(cls, *args, **kwargs):
        return super(MyClass, cls).__new__(cls, *args, **kwargs) # with `cls`
    def __init__(self, *args, **kwargs):
        return super(MyClass, self).__init__(*args, **kwargs) # without `self`

When I compared these functions I got more confused:

>>> cls = object
>>> self = cls()
>>> cls.__new__ is self.__new__
True
>>> cls.__init__ is self.__init__
False
>>> self.__init__()
>>> cls.__init__()
Traceback (most recent call last): ...

So, what are the differences between __new__ and __init__ behind these results? Which methods are bound and which are free? Why are you able to call self.__init__() but not cls.__init__()? Is cls.__init__ a method defined in cls itself or in its metaclass?

Mad Physicist
  • 107,652
  • 25
  • 181
  • 264
Cyker
  • 9,946
  • 8
  • 65
  • 93

3 Answers3

5

The biggest part of the picture you're probably missing is that __new__ is a staticmethod, special-cased to be one even if you don't use the @staticmethod decorator.

When calling a method through super(), super() performs the same kind of argument binding that would be performed normally for that kind of method (using the descriptor protocol). For a staticmethod like __new__, that means no arguments are automatically bound, so cls has to be passed explicitly. For an instance method like __init__, that means self is bound automatically, which is why you don't have to pass self to super().__init__.

user2357112
  • 260,549
  • 28
  • 431
  • 505
  • I was wondering why `__new__` is made a static but not a class method, while you still have to pass a class argument. – Cyker Jan 28 '19 at 04:35
  • @Cyker: With a staticmethod, you can do stuff like `object.__new__(Whatever)`. It'd be a lot more awkward to replicate that functionality with a classmethod. – user2357112 Jan 28 '19 at 04:39
  • But why `object.__new__(Whatever)` while you can `Whatever.__new__()`? – Cyker Jan 28 '19 at 04:42
  • 1
    @Cyker: Because you want to use `object`'s `__new__` method instead of `Whatever`'s. – user2357112 Jan 28 '19 at 04:44
  • Then why don't you want to use `object`'s `__init__` method instead of `Whatever`'s? – Cyker Jan 28 '19 at 09:34
  • @Cyker: `object.__new__(Whatever)` doesn't call any `__init__` method at all. If you want to call `object.__init__`, you can call it manually on the resulting object, but `object.__init__` doesn't perform any initialization. In any case, the usual reason to call a superclass `__new__` manually outside of a subclass `__new__` is because the subclass `__new__` would refuse to create the object. – user2357112 Jan 28 '19 at 16:19
1

cls stands for class itself, while self stands for object itself. These are just conventions.
The __new__ method is called before the object is created, in fact, __new__should create the object and return it. Therefore, it needs a class to create object. After that, the __init__ is called to initialize the object, so it needs the object as the first argument.

For example:

class MyClass:
    def __new__(cls, *args, **kwargs):
        # cls == MyClass
        return super().__new__(cls, *args, **kwargs)
        # cls here is because __new__ is staticmethods so you have to pass the cls explicitly

        # You can't use cls() here because it will call this methods again and again
        # causing recusion error

    def __init__(self, *args, **kwargs):
        # Here self is the instance(or object) of MyClass
        # So you can initialize it by self.xxx
        self.xxx = 'xxx'

__new__ is static method, so the class and instance share the same __new__ method. __init__ is the method of instance. If you want to call it via class, you need to pass the instance as the first argument explicitly.

cls.__init__(self)

Anything in Python is object, including the class itself. So for class, it has its own __new__ and __init__ which are used by metaclass to create class and initialize class.
These are meta programming of Python, I suggest to read the ninth chapter of Python Cookbook.

Hou Lu
  • 3,012
  • 2
  • 16
  • 23
1

The main purpose of __new__ is to allocate a new instance of the class, while __init__'s job is to set up an existing instance.

According to the docs:

__new__() is a static method (special-cased so you need not declare it as such)

__init__ on the other hand, is a proper instance method. It can be called multiple times on the same instance, by the way.

That should be enough to explain your terminal session:

>>> cls = object
>>> self = cls()

You just called object.__call__, which essentially does

self = cls.__new__()
if isinstance(self, cls):
    cls.__init__(self)
return self

Notice that the return value of __new__ is not required to be an instance of the class it belongs to, but __init__ is called only if it is. In your case, it is.

>>> cls.__new__ is self.__new__
True

__new__ is a static method, so attempting to bind it to the instance does nothing: it stays a class method. This is the same reason that you have to pass cls explicitly when calling super().__new__: it's the same free function, unbound to either class or instance.

 >>> cls.__init__ is self.__init__
 False

Not only are these not the same thing, but their types are different. cls.__init__ is a regular function. self.__init__ is a bound method which lacks the first parameter of cls.__init__.

>>> self.__init__()

This has already been called, but for object, it's a no-op you can call as many times as you like. Notice that the first parameter is not being passed in, being as it is a bound method.

>>> cls.__init__()

This is calling the raw __init__ function, which requires that the self parameter be passed in. Since you don't do that, it raises. Try this instead:

>>> cls.__init__(self)
user2357112
  • 260,549
  • 28
  • 431
  • 505
Mad Physicist
  • 107,652
  • 25
  • 181
  • 264
  • If `self.__init__()` invokes `__init__` method defined in `type(self)`, why wouldn't `cls.__init__()` invoke `__init__` method defined in `type(cls)`? Isn't a class object also an object? – Cyker Jan 28 '19 at 05:14
  • @Cyker. Interesting point. Because a method is a non-data descriptor, so any attribute on the instance with the same name will override it. If you do `self.__init__ = some_other_function`, you will see that the other function gets called when you do `self.__init__`. If you hadn't defined `__init__` in the class, it *would* use the one in the metaclass. It's interesting to note that you can't override dunder methods like that on the instance level because dunder methods are always called as `type(self).__dunder__(self, ...)` when used as operators. – Mad Physicist Jan 28 '19 at 05:20
  • 1
    I think if I don't define `__init__` in the class, it would use the one in its superclass. Only if the superclass doesn't have this method it would use the one from its metaclass. But `object.__init__` always exists so the one in its metaclass won't be used. – Cyker Jan 28 '19 at 05:38
  • @Cyker correct. Object does not have a superclass, which simplifies that case though. – Mad Physicist Jan 28 '19 at 05:45