Pybind11: Transfer Class Ownership to C++ on Construct

Question

I'm having an issue where a python class, which is derived from a c++ base class using pybind11, is being immediately destructed (garbage collected). I would like C++ to take ownership of the dynamically allocated object, but I can't seem to make that happen. I've tried keep_alive, passing shared_ptr<> as py::class_ template argument, and py::return_value_policy... nothing is working. I suspect this is just user error.

This is a simplification of the real issue I'm having with a much larger code base that is architected similarly. Changing the architecture is not an option, so making this example work is critical for me.

I have two c++ classes that I have created python interfaces for using pybind11. Class A and B both have virtual methods, so they have corresponding trampoline classes to support inheritance. The user calls the B::Run() function which results in a dynamically allocated (via new) A object to be created. When I create specializations of these two classes in python, as shown below.... Segmentation fault because the B::aBase is destroyed immediately after B::Run being called.

Any Ideas how to fix this? Thanks in advance!

class A
{
public:
    A(){};
    virtual ~A()
    {
        std::cout << "In A::~A()\n";
    };

    virtual char* SayHello()
    {
        char* x = "\n\nHello from Class A\n\n";
        return x;
    }
};

class ATramploline : public A
{
public:
    using A::A;
    char* SayHello() override
    {
        PYBIND11_OVERLOAD( char*,A,SayHello,);
    }
};


class B
{
public:
    B()
    {
        std::cout << "In Class B Constructor\n";
    }

    void Run()
    {
        aBase = AllocateAnAClass();
        std::cout << aBase->SayHello();
    }

    virtual ~B()
    {
        fprintf(stderr,"About to delete aBase");
        delete aBase;
    }

    A* aBase;

    virtual A* AllocateAnAClass()
    {
        return new A;
    }
};

class BTramploline : public B
{
public:
    using B::B;
    A* AllocateAnAClass() override
    {
        PYBIND11_OVERLOAD( A*,B,AllocateAnAClass,);
    }
};

PYBIND11_MODULE(TestModule,m)
{
    py::class_<A,ATramploline>(m,"A")
        .def(py::init<>(),py::return_value_policy::reference_internal)
        .def("SayHello",&A::SayHello);

    py::class_<B,BTramploline>(m,"B")
        .def(py::init<>())
        .def("Run",&B::Run)
        .def("AllocateAnAClass",&B::AllocateAnAClass,py::return_value_policy::reference_internal);
}



#!/usr/bin/python3

from TestModule import A,B
class MyA(A):
    def __init__(self):
        super().__init__()
        print("Done with MyA Constructor")

    def SayHello(self):
        return '\n\nHello from Class MyA\n\n'

class MyB(B):
    def __init__(self):
        super().__init__()
        print("Done With MyB Constructor")
    def AllocateAnAClass(self):
        print("In MyB::AllocateAnAClass!!!")
        return MyA()

#x = B()
#x.Run()

y = MyB()
y.Run()

print("done with test script\n")

What does SWIG have to do with this? – Alex Huszagh Feb 27 '18 at 13:50 — Alex Huszagh, Feb 27 '18 at 13:50
Please provide a minimal but complete example. – Ulrich Eckhardt Feb 27 '18 at 13:52 — Ulrich Eckhardt, Feb 27 '18 at 13:52
This is a minimal but complete example – IAS_LLC Feb 28 '18 at 13:13 — IAS_LLC, Feb 28 '18 at 13:13

score 2 · Answer 1 · answered Feb 27 '18 at 14:57

The correct (I think) way to use std::shared_ptr<A> as the A holder is to add it to class_<A...> arguments.

You also want to replace every instance of A* with std::shared_ptr<A>, and new with std::make_shared. I think non-default return policies are not needed in this case, so I have removed them; YMMV.

Working module below (with minor errors corrected).

#include <pybind11/pybind11.h>
#include <memory>
#include <iostream>

namespace py = pybind11;

class A
{
public:
    A(){};
    A(const A&) { std::cout << "Copying A\n"; }
    virtual ~A()
    {
        std::cout << "In A::~A()\n";
    };

    virtual const char* SayHello()
    {
        const char* x = "\n\nHello from Class A\n\n";
        return x;
    }
};

class ATrampoline : public A
{
public:
    using A::A;
    const char* SayHello() override
    {
        PYBIND11_OVERLOAD( const char*,A,SayHello,);
    }
};


class B
{
public:
    B()
    {
        std::cout << "In Class B Constructor\n";
    }

    B(const B&) { std::cout << "Copying B\n"; }
    void Run()
    {
        aBase = AllocateAnAClass();
        std::cout << aBase->SayHello();
    }

    virtual ~B()
    {
    }

    std::shared_ptr<A> aBase;

    virtual std::shared_ptr<A> AllocateAnAClass()
    {
        return std::make_shared<A>();
    }
};

class BTrampoline : public B
{
public:
    using B::B;
    std::shared_ptr<A> AllocateAnAClass() override
    {
        PYBIND11_OVERLOAD(std::shared_ptr<A>,B,AllocateAnAClass,);
    }
};

PYBIND11_MODULE(TestModule,m)
{
    py::class_<A,std::shared_ptr<A>, ATrampoline>(m,"A")
        .def(py::init<>())
        .def("SayHello",&A::SayHello);

    py::class_<B, BTrampoline>(m,"B")
        .def(py::init<>())
        .def("Run",&B::Run)
        .def("AllocateAnAClass",&B::AllocateAnAClass);
}

score 2 · Accepted Answer · answered Feb 28 '18 at 13:21

py::nodelete was the solution. While n.m's answer DOES work, it would require going back and chaning all of the pointer in an existing libary to smart pointers, which isn't a viable option for me. Using py::nodelete allows me to do everything on the pybind11 side.

py::class_<A,ATramploline,std::unique_ptr<A,py::nodelete> >(m,"A")
        .def(py::init<>())
        .def("SayHello",&A::SayHello);

Pybind11: Transfer Class Ownership to C++ on Construct

2 Answers2

Linked