1

I'm trying to use pybind11 to bind heavy computing in C++ with python as front end (unfortunately). I need to pass string/bytes around from python to C++ and back. After reading through the documentation, I thought that "without conversion" meaning that the memory address of C++ and python should be the same when passing around, they are looking at the same data. Here is the dummy code:

In c++:

void string_fn(std::string& x) {
    std::cout << "c++ value is:  " << x << std::endl;
    std::cout << "c++ address is: " << &x << std::endl;
}

PYBIND11_MODULE(module_name, module_handle) {
  module_handle.def("string_fn", &string_fn, py::return_value_policy::reference);
}

Pass data from python:

a = "test_string" # same output with a = b"test_string"
print("python address: ", hex(id(a)))

# call c++ function
string_fn(a)

Outputs are:

python address:  0x7f268c65f240
c++ value is:  test_string
c++ address is: 0x7ffff24c1830

As we can see, the value is the same but the address is different. I'm a total noob and my questions are:

  1. Am I printing the right thing? Or pybind11 just create a copy of python string?
  2. If I'm doing it wrong, how should I achieve it with zero copy?
npsable
  • 13
  • 3
  • 1
    I do not understand how the title is related to the question – 463035818_is_not_an_ai Jul 18 '23 at 07:34
  • I guess "without conversion" mentioned in "Passing bytes to C++" means that python will not convert `py::bytes` to `py::str`. About pythonic `id` being the same as transformed argument to c++ function - I don't see anywhere any mentions about it. – pptaszni Jul 18 '23 at 07:37
  • "string/bytes" String **or** bytes? These are totally different things. – n. m. could be an AI Jul 18 '23 at 07:42
  • Hi guys, I'm sorry for the confusion. I'm just trying to pass data from python to c++ without it making a copy. Following the docs, I thought that "without conversion" meant that c++ and python should point to the same data, with no need to convert to c++ type. Is there anywhere in the doc I should look into to achieve this? – npsable Jul 18 '23 at 08:22
  • https://stackoverflow.com/questions/57990269/passing-pointer-to-c-from-python-using-pybind11 – n. m. could be an AI Jul 18 '23 at 11:04
  • I'd say there's definitely a copy involved to be able to get a `std::string&` since a) without that you would be able to modify the immutable Python string and b) AFAIK `std::string` owns its data, and there's no (normal) way to make a non-owning one. | What is the bigger problem you're trying to solve? What sort of data is it, what do you want to do with it exactly? | For example OpenCV uses numpy arrays on Python side to facilitate passing large mutable arrays back and forth efficiently. Pybind11 has support for those. – Dan Mašek Jul 19 '23 at 10:47

0 Answers0