Initialize PyObject without calling init

Question

I have a wrapper class as in the example below, written in Python.

class Foo:
    def __init__(self, bar=None):
        if bar:
            self._bar = bar
        else:
            self._bar = CreateBarObject("value")

I create an instance via the Python C API

// c++ pseudo code for convenience

auto obj = PyObject_CallObject(Foo) 
auto bar = CreateBarObject("another_value");
PyObject_SetAttrString(obj, "_bar", bar)

As you can see from the code, Foo.__init__ will be called when an instance gets created which creates a new bar object. But I would like to bypass this "heavy" operation. So any safe way to create an instance of Foo so I can set self._bar via the Python C API? Any ideas?

Why do you want to set `_bar` through the C API? Why not just pass the `bar` object to `Foo` in the initial call? — user2357112, Jul 09 '18 at 01:36
`Foo()` with no args needs to create `self._bar` to make the instance work. But this call is a very expensive operation so I would like to bypass this cost plus bypass unnecessary byte code instructions, it needs to be a very lightweight initialization if possible — Daniel Stephens, Jul 09 '18 at 01:40
"`Foo()` with no args needs to create `self._bar` to make the instance work" - yes, so *why are you passing it no arguments*? — user2357112, Jul 09 '18 at 01:51
Because calling `__init__` and `if bar` have a significant performance impact for at least this class, so I try to optimize it. Btw, you were right, data was supposed to be `bar`. Thanks for pointing it out — Daniel Stephens, Jul 09 '18 at 02:06
Not really relevant to the actual question but are you writing a C++ extension or a C extension? Because `auto` seems very much like C++ because it would be weird to [use `auto` in C](https://stackoverflow.com/q/2192547). — MSeifert, Jul 09 '18 at 06:48

score 4 · Accepted Answer · answered Jul 09 '18 at 02:15

You should be able to directly invoke the tp_new of Foo (equivalent to invoking Foo.__new__ at the Python layer). That will perform the allocation and C level "mandatory initialization" work, without the "optional initialization" work of tp_init/__init__. It's the same strategy pickle uses to create instances of a type without initializing them (so it can fill them in via __setstate__ or directly filling in __dict__, as appropriate).

Assuming Foo is a C level PyTypeObject*, something like this should do the trick:

auto emptytup = PyTuple_New(0);  /* AFAICT, the args tuple is mandatory, but kwargs is optional */
/* Error check for PyTuple_New failure */
auto obj = Foo->tp_new(Foo, emptytup, NULL);
Py_DECREF(emptytup);
/* Error check for new fail */
auto bar = CreateBarObject("another_value");
PyObject_SetAttrString(obj, "_bar", bar)

score -1 · Answer 2 · edited Jul 09 '18 at 02:03

-1

Make self.bar a property that creates the instance the first time it's needed.

class Foo:

    def __init__(self, bar=None):
        self._bar = bar

    @property
    def bar(self):
        if self._bar is None:
            self._bar = CreateBarObject(...)
        return self._bar

The instance is fully functional. The instance's bar property will create the Bar object the first time it's needed. Or you can set _bar from the C API before that, and that'll be used instead. Or you can pass in an existing Bar object when instantiating the class. Your choice.

edited Jul 09 '18 at 02:03

Jonathon Reinhart

132,704
33
254
328

answered Jul 09 '18 at 01:57

kindall

178,883
35
278
309

Because calling `__init__` and `if bar` have a significant performance impact for at least this class I would like to get rid of the Python byte code execution in this case. But for a general approach, this is a really nice idea, never thought about that – Daniel Stephens Jul 09 '18 at 02:07
You could just do away with `__init__` then and put an class attribute `_bar = None`. That will do away with all bytecode at `Foo` initialization. – kindall Jul 09 '18 at 02:18
Foo() needs to still have a `_bar` member with CreateCreateObject("value"). But I think I found a solution ShadowRanger posted. Thanks a lot for your input! – Daniel Stephens Jul 09 '18 at 02:35
You've now moved the performance problem to *each* `bar` access, not just the constructor. In this case this should at least be a lazy/reify non-data descriptor. – Antti Haapala -- Слава Україні Jul 09 '18 at 05:50

Initialize PyObject without calling __init__

2 Answers2

Initialize PyObject without calling init