__new__ and __init__ in Python

Question

I am learning Python and so far I can tell the things below about __new__ and __init__:

1. __new__ is for object creation
2. __init__ is for object initialization
3. __new__ is invoked before __init__ as __new__ returns a new instance and __init__ invoked afterwards to initialize inner state.
4. __new__ is good for immutable object as they cannot be changed once they are assigned. So we can return new instance which has new state.
5. We can use __new__ and __init__ for both mutable object as its inner state can be changed.

But I have another questions now.

1. When I create a new instance such as a = MyClass("hello","world"), how these arguments are passed? I mean how I should structure the class using __init__ and __new__ as they are different and both accepts arbitrary arguments besides default first argument.
2. self keyword is in terms of name can be changed to something else? But I am wondering cls is in terms of name is subject to change to something else as it is just a parameter name?

I made a little experiments as such below:

>>> class MyClass(tuple):
    def __new__(tuple):
        return [1,2,3]

and I did below:

>>> a = MyClass()
>>> a
[1, 2, 3]

Albeit I said I want to return tuple, this code works fine and returned me [1,2,3]. I knew we were passing the first parameters as the type we wanted to receive once the __new__ function is invoked. We are talking about New function right? I don't know other languages return type other than bound type?

And I did anther things as well:

>>> issubclass(MyClass,list)
False
>>> issubclass(MyClass,tuple)
True
>>> isinstance(a,MyClass)
False
>>> isinstance(a,tuple)
False
>>> isinstance(a,list)
True

I didn't do more experiment because the further wasn't bright and I decided to stop there and decided to ask StackOverflow.

The SO posts I read:

1. Python object creation
2. Python's use of __new__ and __init__?

Answer 1

how I should structure the class using __init__ and __new__ as they are different and both accepts arbitrary arguments besides default first argument.

Only rarely will you have to worry about __new__. Usually, you'll just define __init__ and let the default __new__ pass the constructor arguments to it.

self keyword is in terms of name can be changed to something else? But I am wondering cls is in terms of name is subject to change to something else as it is just a parameter name?

Both are just parameter names with no special meaning in the language. But their use is a very strong convention in the Python community; most Pythonistas will never change the names self and cls in these contexts and will be confused when someone else does.

Note that your use of def __new__(tuple) re-binds the name tuple inside the constructor function. When actually implementing __new__, you'll want to do it as

def __new__(cls, *args, **kwargs):
    # do allocation to get an object, say, obj
    return obj

Albeit I said I want to return tuple, this code works fine and returned me [1,2,3].

MyClass() will have the value that __new__ returns. There's no implicit type checking in Python; it's the responsibility of the programmer to return the correct type ("we're all consenting adults here"). Being able to return a different type than requested can be useful for implementing factories: you can return a subclass of the type requested.

This also explains the issubclass/isinstance behavior you observe: the subclass relationship follows from your use of class MyClass(tuple), the isinstance reflects that you return the "wrong" type from __new__.

For reference, check out the requirements for __new__ in the Python Language Reference.

Edit: ok, here's an example of potentially useful use of __new__. The class Eel keeps track of how many eels are alive in the process and refuses to allocate if this exceeds some maximum.

class Eel(object):
    MAX_EELS = 20
    n_eels = 0

    def __new__(cls, *args, **kwargs):
        if cls.n_eels == cls.MAX_EELS:
            raise HovercraftFull()

        obj = super(Eel, cls).__new__(cls)
        cls.n_eels += 1
        return obj

    def __init__(self, voltage):
        self.voltage = voltage

    def __del__(self):
        type(self).n_eels -= 1

    def electric(self):
        """Is this an electric eel?"""
        return self.voltage > 0

Mind you, there are smarter ways to accomplish this behavior.

Answer 2

Your question was something like,

What is the difference between __new__ and __init__ in Python?

---

Suppose that we have a very simple class named Rectangle

class Rectangle():
    def __init__(self, width:float, height:float):
        self._width = width
        self._hieght = height

We instantiate the Rectangle class as follows:

rocky_the_rectangle = Rectangle(10.01, 8.91)

In order to help explain what __new__ does, I will write a classmethod named make_rectangle which has almost the same behavior as the class-constructor.

class Rectangle():
    def __new__(cls, *args, **kwargs):
        print("__new__(" + ", ".join(str(x) for x in [cls, *args]) + ")")
        obj = super().__new__(cls)
        return obj

    def __init__(self, width:float, height:float):
        args = (width, height)
        print("__init__(" + ", ".join(str(x) for x in [self, *args]) + ")")
        self._width = width
        self._height = height

    @classmethod
    def make_rectangle(cls, *args):
        new_instance = cls.__new__(cls, *args)
        if isinstance(new_instance, cls):
            cls.__init__(new_instance, *args)
        return new_instance

Now we can instantiate the Rectangle class as follows:

rocky_II = Rectangle.make_rectangle(10.01, 8.91)

The __init__ method is unable to replace the self parameter with something new and different.

For example, people often want to put a wrapper around the self parameter. You might want something like this:

import functools
import sys

class decorator:

    def __new__(cls, kallable):
        instance = super().__new__(cls)
        instance = functools.update_wrapper(instance, kallable)
        return instance

    def __init__(self, kallable):
        self._kallable = kallable
        self._file     = sys.stdout

    def __call__(self, *args, **kwargs):
        print("__init__(" + ", ".join(str(x) for x in [self, *args]) + ")", file=self._file)
        return self._kallable(*args, **kwargs)

@decorator
def pow(base:float, exp:int):
    """
         +------------------------------------------+
         |                EXAMPLES                  |
         +------------------------------------------+
         | BASE | EXPONENT |       OUTPUT           |
         +------+----------+------------------------+
         |    2 |        5 | 2^5      |          32 |
         |  2.5 |        7 | 2.5^7    | 610.3515625 |
         |   10 |        3 | 10^3     |        1000 |
         |  0.1 |        5 | 0.1^5    |     0.00001 |
         |    7 |        0 | 7^0      |           1 |
         +------+----------+----------+-------------+
    """
    base = float(base)
    # convert `exp` to string to avoid flooring, or truncating, floats
    exp  = int(str(exp))
    if exp > 0:
        return base * pow(base, exp-1)
    else: # exp == 2
        return 1

result1 = pow(2, 5)
result2 = pow(8.1, 0)
print("pow(2, 5) == "  , result1)
print("pow(8.1, 0) == ", result2)

print("What is the documentation string? The doc-string is... ", pow.__doc__)

It will not help to use functools.update_wrapper inside of __init__. If you try to write the following...

class f:
    def __init__(outter, inner):
        # `outter` is usually named `self`, but you are...
        #     ... allowed to use other names for it.  
        outter = functools.update_wrapper(outter, inner) 

... then outter will get ignored. You cannot replace the parameter named self with a different self

__new__ allows us to replace self with a wrapper around self

class decorator:

    def __new__(cls, kallable):
        instance = super().__new__(cls)
        instance = functools.update_wrapper(instance, kallable)
        return instance

Without using functools.update_wrapper inside of __new__ the doc-string inside of the original callable will be wiped-out, ignored, not inherited, and/or shadowed.

Answer 3

seems that nobody had cover how the arguments are handled through the instantiation

lets build a simple class that define new and init

censured = ["Cicciogamer"]

class Foo(object):
    def __new__(cls, name):
        if name in censured:
            print("you shouldn't do this")
        return super().__new__(cls)

    def __init__(self, var):
        self.var = var

when you call a class object to get an instance, python implicity call

Foo.__call__(*args, **kwargs)

so with the class above you get the (maybe) undesired double arguments passing:

foo = Foo("Cicciogamer")
>>> "you shouldn't do this"
foo.var
>>> "Cicciogamer"
    

To achieve the control on how these parameters behave ASFAIK you must override the call method of the CLASS OBJECT not instance

you can use metaclasses but it might be overkill

class MetaFoo:
    def __call__(cls, name, var):
        foo_object = cls.__new__(name)
        cls.__init__(foo_object, var)
        return foo_object

class Foo(metaclass=MetaFoo):
    ...


foo = Foo("Cicciogamer", 505)
>>> "you shouldn't do this"
foo.var
>>> 505

or i think you can achieve this with simply

class Foo:
    ...

    @classmethod
    def new(cls, name, var):
        foo_object = cls.__new__(name)
        cls.__init__(foo_object, var)
        return foo_object

    __call__ = new

please let me notice if this could be achieved in a better way, or if i simply screwed up something