Is it possible to assign an C++ pointer to an attribute of a Python object, and retrieve it as a C++ pointer in another C++ function?

Question

I'm making a C++ extension for Python, and I'm trying to do something like:

// this function assigns a C++ pointer to as attribute of a python object
void function1(PyObject* p){
    // equivalent of p.attr = cpp_attr;
    MyClass* cpp_attr = new MyClass();
    PyObject* args = PyTuple_Pack(cpp_attr);
    PyObject_SetAttrString(p, (char*)"attr", args);
}

I would like to retrieve this pointer and set it as attribute of another C++ object. I know how to get the PyObject* but after that I'm not sure what to do anymore

MySecondClass::MySecondClass(PyObject* p){
    // get the attribute from p; equivalent of cpp_attr = p.attr
    PyObject* cpp_attr = PyObject_getAttrString(p, (char*)"attr"));
    // somehow get back the pointer to MyClass object created in function1
    
}

I looked at the documentation but I couldn't find anything that returns the original type. Is there anyway to do this?

Thanks

Is `MyClass` a Python object? Otherwise what you're trying to do is completely wrong. — DavidW, Jun 27 '20 at 06:53

DavidW · Answer 1 · 2020-06-28T21:19:59.223

It's difficult to be absolutely certain, but I doubt that MyClass a Python object. This means that your attempt to store it as a Python object (e.g. using PyTuple_Pack) is completely wrong and will cause Python to malfunction in unexpected ways.

What will happen is that Python will attempt to interpret the pointer as a Python object, will try to use its normal reference counting mechanisms on that object (will change it in unpredictable ways), and ultimately try to deallocate that object (using Python mechanisms, not delete...) if some part of the object happens to equal 0.

There's a number of options, all basically centred around creating a wrapper object - a Python object defined in C++ that holds either a pointer or value of your C++ object.

Do it manually using the Python C API - This answer gives a very thorough example.

Look up the PyCapsule interface to create a quick wrapper around your object. You'd create your capsule with:

 PyObject* cap = PyCapsule_New(cpp_attr, "MyClass",
       [](PyObject* c) {
           auto deleteme = reinterpret_cast<MyClass*>(PyCapsule_GetPointer(c, "MyClass));
           delete deleteme;
       });

And you retrieve your C++ class from the capsule with:

 reinterpret_cast<MyClass*>(PyCapsule_GetPointer(c, "MyClass))

Use some tool like PyBind11, Cython, SWIG, etc to create the wrapper object for you.

Note also that PyObject_SetAttrString does not require the third argument to be a tuple (unless you specifically want to store a tuple...). You're likely getting it confused with PyObject_Call, where the args are passed as a tuple.

thanks for the link. So basically, everything in a PyTuple_Pack has to be a PyObject* instead of a C++ pointer? — qwerty_99, Jun 28 '20 at 20:32
Yes. Everything _must_ be a `PyObject*`. Also, the first argument should be an integer telling it how many PyObject's follow (which I missed on first look). https://docs.python.org/3/c-api/tuple.html#c.PyTuple_Pack — DavidW, Jun 28 '20 at 21:19

zhanga · Accepted Answer · 2020-06-26T19:14:05.243

-1

Assuming your call to PyTuple_Pack is correct, then you've created a PyTupleObject which has a structure:

typedef struct {
  PyObject_VAR_HEAD
  PyObject *ob_item[1];
} PyTupleObject;

The PyTupleObject inherits from the generic PyObject struct which has the following members:

struct _object *_ob_next;
struct _object *_ob_prev;
Py_ssize_t ob_refcnt;
struct _typeobject *ob_type;

You can access the latter two with the macrosPy_REFCNT and Py_TYPE

The ob_item[1] member should be a pointer to the memory initially allocated. Based on how Macros are written in the documentation, you should be able to access it by

((PyTupleObject *)cpp_attr)->ob_item

And if you know the data type of the C++ pointer, then you should be able to cast it back. Maybe you can try

MyClass* cpp_att_again = reinterpret_cast<MyClass*>((PyTupleObject *)cpp_attr)->ob_item

Hopefully this points you in the right direction. You might be able to glean more insight from a similar question.

edited Jun 26 '20 at 19:14

answered Jun 26 '20 at 19:04

zhanga

89
2
10

Thanks! I've read that PyTuple_Pack was required to pass arguments to python functions, is this correct? – qwerty_99 Jun 26 '20 at 19:07
I don't think that's the right way to put it. A python function would take in a python object as an argument. So as long as you've successfully created the appropriate python object to point to your C++ objects/variables, then you can pass those in to the python function. That said, I'm not sure if the tuple type is the right choice. If `cpp_attr` was an array, it might be the right thing to do. If it's a class with member functions, you probably need a different approach. – zhanga Jun 26 '20 at 19:30
I take that back. If you're calling a python function from C++, using a tuple is appropriate to format the input arguments. For completeness, it might be useful to describe how this `cpp_attr` is going to be used in the Python-side. – zhanga Jun 26 '20 at 19:41
it's only for storing some integer values, I don't know all the details though – qwerty_99 Jun 26 '20 at 19:46
Why not use [`PyTuple_GetItem`](https://docs.python.org/3/c-api/tuple.html#c.PyTuple_GetItem) (the _actual_ interface for this) instead of messing around with the tuple internals? – DavidW Jun 27 '20 at 07:36

Is it possible to assign an C++ pointer to an attribute of a Python object, and retrieve it as a C++ pointer in another C++ function?

2 Answers2