PyBind11: boost::multiprecision::cpp_int to Python

Question

I am interested in using pybind11 to optimize some Python computation using C++. The casting documentation doesn't make much sense to me and was wondering if anyone knew how to cast boost datatypes, specifically cpp_int, to a Python datatype so I can return computations. A simple example of what I'm trying to do would be factorials:

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>

#include <boost/multiprecision/cpp_int.hpp>


using boost::multiprecision::cpp_int;
namespace py = pybind11;

py::int_ fact(int i) {
    cpp_int prod = 1;
    while(i-- >= 1){
        prod *= (i+1);
    }
    return py::cast(prod);
}

PYBIND11_MODULE(fact, m) {
  m.def("fact", &fact,R"pbdoc(
        Returns the factorial of a number.
    )pbdoc");
}

The above compiles but when I go to use it I get

TypeError: Unregistered type : boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<0u, 0u, (boost::multiprecision::cpp_integer_type)1, (boost::multiprecision::cpp_int_check_type)0, std::allocator<unsigned long long> >, (boost::multiprecision::expression_template_option)1>

so something isn't working with py::cast I think...
My laptop runs Windows 10 and I am using Anaconda Python 3.7

C:\Users\15734>python
Python 3.7.1 (default, Dec 10 2018, 22:54:23) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.

and Stephan T. Lavavej's MinGW C++ 8.2.0

C:\Users\15734>g++ -v
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=c:/mingw/bin/../libexec/gcc/x86_64-w64-mingw32/8.2.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../src/configure --enable-languages=c,c++ --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --disable-multilib --prefix=/c/temp/gcc/dest --with-sysroot=/c/temp/gcc/dest --disable-libstdcxx-pch --disable-libstdcxx-verbose --disable-nls --disable-shared --disable-win32-registry --with-tune=haswell --enable-threads=posix --enable-libgomp
Thread model: posix
gcc version 8.2.0 (GCC)

I am also using Build Tools for Visual Studio 2017 for the vcruntime140.dll located in "C:\ProgramFiles(x86)\MicrosoftVisualStudio\2017\BuildTools\VC\Redist\MSVC\14.16.27012\onecore\x64\Microsoft.VC141.CRT" (which I copied and pasted into "C:\MinGW\lib"). I also changed all string "gcc" in "C:\Anaconda3\Lib\distutils\cygwinccompiler.py" to "g++" (I did not change the variable names with gcc in them).

Keywords:
"pybind11" ; "Boost" ; "C++ and Python" ; "boost::multiprecision::cpp_int"

cpp_int (for some backends) is just a wrapper around a C library (mpz, maybe?) So it might be easier just to wrap the C directly. — user14717, Feb 18 '19 at 02:18
@user14717 If I wrapped mpz how would I use it in C++ and return it via pybind11? — Grant, Feb 18 '19 at 02:23
Well, you wouldn't use it in C++ at all. If you need to use it in both C++ and Python, my suggestion won't work. — user14717, Feb 18 '19 at 02:25
If you go directly from C->Python, you'd build a python extension module. — user14717, Feb 18 '19 at 02:25
@user14717 Do you know of any good resources for the C extension module? — Grant, Feb 18 '19 at 02:28
Maybe you can try to replace [`struct inty`](https://pybind11.readthedocs.io/en/stable/advanced/cast/custom.html) with `cpp_int`?. You could convert cpp_int to base16 string and pass it to [PyLong_FromString](https://docs.python.org/3/c-api/long.html#c.PyLong_FromString) to convert from C++-->Python, and Python-->C++ by casting PyNumber to string base16 with [PyNumber_ToBase](https://docs.python.org/3/c-api/number.html?highlight=pynumber_long#c.PyNumber_ToBase) and construct a cpp_int based on this Python string object. — tnt, Feb 18 '19 at 04:47
@tntxtnt Does boost come with a cpp_int to base16 converter? I failed to find a converter via Google... — Grant, Feb 18 '19 at 05:02
You could use ostringstream oss and iomanip to output cpp_int as hex `oss << std::hex << bigint` then get its `oss.str()` — tnt, Feb 18 '19 at 13:13

tnt · Accepted Answer · 2019-02-18T17:04:06.870

I got this working:

#include <boost/multiprecision/cpp_int.hpp>
#include <iomanip>
#include <pybind11/pybind11.h>
#include <sstream>

using cpp_int = boost::multiprecision::cpp_int;
namespace py = pybind11;

namespace pybind11
{
namespace detail
{
    template <>
    struct type_caster<cpp_int> {
        /**
         * This macro establishes the name 'cpp_int' in
         * function signatures and declares a local variable
         * 'value' of type cpp_int
         */
        PYBIND11_TYPE_CASTER(cpp_int, _("cpp_int"));

        /**
         * Conversion part 1 (Python->C++): convert a PyObject into a cpp_int
         * instance or return false upon failure. The second argument
         * indicates whether implicit conversions should be applied.
         */
        bool load(handle src, bool)
        {
            // Convert into base 16 string (PyNumber_ToBase prepend '0x')
            PyObject* tmp = PyNumber_ToBase(src.ptr(), 16);
            if (!tmp) return false;

            std::string s = py::cast<std::string>(tmp);
            value = cpp_int{s}; // explicit cast from string to cpp_int,
                                // don't need a base here because
                                // `PyNumber_ToBase` already prepended '0x'
            Py_DECREF(tmp);

            /* Ensure return code was OK (to avoid out-of-range errors etc) */
            return !PyErr_Occurred();
        }

        /**
         * Conversion part 2 (C++ -> Python): convert an cpp_int instance into
         * a Python object. The second and third arguments are used to
         * indicate the return value policy and parent object (for
         * ``return_value_policy::reference_internal``) and are generally
         * ignored by implicit casters.
         */
        static handle cast(const cpp_int& src, return_value_policy, handle)
        {
            // Convert cpp_int to base 16 string
            std::ostringstream oss;
            oss << std::hex << src;
            return PyLong_FromString(oss.str().c_str(), nullptr, 16);
        }
    };
} // namespace detail
} // namespace pybind11

py::int_ fact(int i)
{
    cpp_int prod = 1;
    while (i-- > 1) prod *= i + 1;
    return py::cast(prod);
}

PYBIND11_MODULE(python_example, m)
{
    m.def("fact", &fact, R"pbdoc(
        Returns the factorial of a number.
    )pbdoc");
}

Probably not the fastest way since it converts cpp_int to a temporary base 16 string first and then convert this string to Python integer.

score 0 · Answer 2 · answered Jul 19 '22 at 16:14

0

I think boost::multiprecision::backends::cpp_int_backend<0u, 0u,... is wrong, you just defined a 0 digit min/max number type. Further it seem cpp_int only parse from string which is rather logical for a multiprecision number.

answered Jul 19 '22 at 16:14

Gold

136
6

PyBind11: boost::multiprecision::cpp_int to Python

2 Answers2

Linked