Exposing a C++ class instance to a python embedded interpreter

Question

I am looking for a simple way to expose a C++ class instance to a python embedded interpreter.

• I have a C++ library. This library is wrapped (using swig for the moment) and I am able to use it from the python interpreter
• I have a C++ main program which instanciates a Foo class from my library and embeds a python interpreter

I would like to expose my C++ world instance of Foo to the python world (and seen as a Foo class).

Is this possible, if so, how?

I think it's almost like in the first answer of : boost::python::ptr or PyInstance_New usage

I guess this means I should use boost.Python to wrap my library?

My only goal is to manipulate my C++ instance of Foo in the embedded python interpreter (not sure that it can be done with the previous method).

In fact, I already have exposed my Foo class to python (with swig).

What I have:

my Foo class:

class Foo{...};

my wrapped library (including the Foo class) exposed to python: so I can start the python interpreter and do something like this :

import my_module
foo=my_modulde.Foo()

What I want:

Having a C++ main program which embeds a python interpreter and manipulates C++ world variables.

int main(int argc, char **argv)
{
    Foo  foo;   // instanciates foo
    
    Py_Initialize();

    Py_Main(argc, argv); // starts the python interpreter
                         // and manipulates THE foo instance in it

    Py_Finalize();
    
    return 0;
}

Answer 1

Boost python Allows you to expose c++ classes to python in a very tightly integrated way - you can even wrap them so that you can derive python classes from your c++ ones, and have virtual methods resolved to the python overrides.

The boost python tutorial is a good place to start.

---

_edit:

You can create a c++ object and pass a reference to it to an internal python interpreter like this:

#include <boost/shared_ptr.hpp>
#include <boost/make_shared.hpp>
#include <boost/python.hpp>
#include <string>
#include <iostream>

namespace bp = boost::python;

struct Foo{
    Foo(){}
    Foo(std::string const& s) : m_string(s){}
    void doSomething() {
        std::cout << "Foo:" << m_string << std::endl;
    }
    std::string m_string;
};

typedef boost::shared_ptr<Foo> foo_ptr;

BOOST_PYTHON_MODULE(hello)
{
    bp::class_<Foo, foo_ptr>("Foo")
        .def("doSomething", &Foo::doSomething)
    ;
};

int main(int argc, char **argv)
{
    Py_Initialize();
    try {
        PyRun_SimpleString(
            "a_foo = None\n"
            "\n"
            "def setup(a_foo_from_cxx):\n"
            "    print 'setup called with', a_foo_from_cxx\n"
            "    global a_foo\n"
            "    a_foo = a_foo_from_cxx\n"
            "\n"
            "def run():\n"
            "    a_foo.doSomething()\n"
            "\n"
            "print 'main module loaded'\n"
        );

        foo_ptr a_cxx_foo = boost::make_shared<Foo>("c++");

        inithello();
        bp::object main = bp::object(bp::handle<>(bp::borrowed(
            PyImport_AddModule("__main__")
        )));

        // pass the reference to a_cxx_foo into python:
        bp::object setup_func = main.attr("setup");
        setup_func(a_cxx_foo);

        // now run the python 'main' function
        bp::object run_func = main.attr("run");
        run_func();
    }
    catch (bp::error_already_set) {
        PyErr_Print();
    }

    Py_Finalize();

    return 0;
}

Answer 2

For reference, here is how you can achieve this using pybind11:

#include <iostream>
#include <pybind11/pybind11.h>
namespace py = pybind11;

// Define C++ class "Foo"
class Foo {
    std::string s_;
public:
    Foo(const std::string &s) : s_(s) {}
    void doSomething() { std::cout << s_ << std::endl; }
};
typedef std::shared_ptr<Foo> FooPtr;

// Define Python module "bar" and Python class "bar.Foo" wrapping the C++ class
PYBIND11_MODULE(bar, m) {
    py::class_<Foo, FooPtr>(m, "Foo")
        .def("doSomething", &Foo::doSomething);
}

int main(int argc, char **argv)
{
    // Create a C++ instance of Foo
    FooPtr foo = std::make_shared<Foo>("Hello, World!");

    // Initialize Python interpreter and import bar module
    PyImport_AppendInittab("bar", PyInit_bar);
    Py_Initialize();
    PyRun_SimpleString("import bar");

    // Make C++ instance accessible in Python as a variable named "foo"
    py::module main = py::module::import("__main__");
    main.attr("foo") = foo;

    // Run some Python code using foo
    PyRun_SimpleString("foo.doSomething()");

    // Finalize the Python interpreter
    Py_Finalize();
    return 0;
}

Answer 3

I know this is an old question, but here is a solution using SWIG.

foo.h:

#pragma once
#include <string>

struct Foo{
  Foo();
  Foo(std::string const& s);
  void doSomething();
  std::string m_string;
};

foo.cpp:

#include "foo.h"
#include <iostream>

Foo::Foo() {}

Foo::Foo(std::string const& s) : m_string(s) {}

void Foo::doSomething() {
  std::cout << "Foo:" << m_string << std::endl;
}

foo.i:

%module module
%{
  #include "foo.h"
%}

%include "std_string.i"
%include "foo.h"

Generate the usual SWIG wrapper together with a runtime

swig -python -c++ -Wall foo.i
swig -python -c++ -Wall -external-runtime runtime.h

Generate the SWIG module containing struct Foo:

g++ -fPIC -Wall -Wextra -shared -o _module.so foo_wrap.cxx foo.cpp -I/usr/include/python2.7 -lpython2.7

If you want to share type information across multiple modules, an argument -DSWIG_TYPE_TABLE=SomeName can be added.

Now, here is how a C++ instance of Foo is passed to the interpreter

#include "foo.h"
#include <Python.h>
#include "runtime.h"

int main(int argc, char **argv) {
  Py_Initialize();

  PyObject* syspath = PySys_GetObject((char*)"path");
  PyObject* pName = PyString_FromString((char*) ".");
  int err = PyList_Insert(syspath, 0, pName);
  Py_DECREF(pName);

  err = PySys_SetObject((char*) "path", syspath);

  PyObject *main, *module, *pInstance, *run, *setup;

  try {
    main = PyImport_ImportModule("__main__");
    err = PyRun_SimpleString(
        "a_foo = None\n"
        "\n"
        "def setup(a_foo_from_cxx):\n"
        "    print 'setup called with', a_foo_from_cxx\n"
        "    global a_foo\n"
        "    a_foo = a_foo_from_cxx\n"
        "\n"
        "def run():\n"
        "    a_foo.doSomething()\n"
        "\n"
        "print 'main module loaded'\n");

    // Load Python module
    module = PyImport_ImportModule("module");

    swig_type_info *pTypeInfo = nullptr;
    pTypeInfo = SWIG_TypeQuery("Foo *");

    Foo* pFoo = new Foo("Hello");
    int owned = 1;
    pInstance =
        SWIG_NewPointerObj(reinterpret_cast<void*>(pFoo), pTypeInfo, owned);

    setup = PyObject_GetAttrString(main, "setup");

    PyObject* result = PyObject_CallFunctionObjArgs(setup, pInstance, NULL);
    Py_DECREF(result);

    run = PyObject_GetAttrString(main, "run");

    result = PyObject_CallFunctionObjArgs(run, NULL);
    Py_DECREF(result);
  }
  catch (...) {
    PyErr_Print();
  }

  Py_DECREF(run);
  Py_DECREF(setup);
  Py_DECREF(pInstance);
  Py_DECREF(module);
  Py_DECREF(main);

  Py_Finalize();
  return 0;
}

The above can be compiled by:

g++ -Wall -Wextra -I/usr/include/python2.7 main.cpp foo.cpp -o main -lpython2.7

Answer 4

2023 answer

It seems that pybind11 makes now this really easy, they have a special header for using pybind for embedding purposes and some detailed documentation:

https://pybind11.readthedocs.io/en/stable/advanced/embedding.html

pybind11 is a header only library, so this is cross-platform and does not add extra dependencies (except for the python interpreter)

Here is the code that solves the original post problem:

#include <pybind11/embed.h>
#include <iostream>

struct Foo{
    Foo(const std::string & s) : m_string(s){}
    void doSomething() {
        std::cout << "Foo:" << m_string << std::endl;
    }
    std::string m_string;
};
typedef std::shared_ptr<Foo> FooPtr;

namespace py = pybind11;

PYBIND11_EMBEDDED_MODULE(bar, m) {
    py::class_<Foo, FooPtr>(m, "Foo")
        .def("doSomething", &Foo::doSomething);
}

int main(int argc, char **argv)
{
    py::scoped_interpreter guard{};
    FooPtr foo = std::make_shared<Foo>("Hello, World!");

    py::module::import("bar");
    py::module main = py::module::import("__main__");
    main.attr("foo") = foo;

    // Run some Python code using foo
    py::exec("foo.doSomething()");
}

Here is a basic CMakeLists.txt that can be used to setup in a cross-platform way a project that embeds python, pybind11 is expected to be in the external subfolder and your source is expected to be named test.cpp:

cmake_minimum_required(VERSION 3.19)
project(test_embed)
find_package(Python3 COMPONENTS Development)
set(CMAKE_CXX_STANDARD 17)

include_directories(extern/pybind11/include ${Python3_INCLUDE_DIRS})
add_executable(test test.cpp)
target_link_options(test PRIVATE ${Python3_LINK_OPTIONS})
target_link_libraries(test PRIVATE ${Python3_LIBRARIES})