Workaround for PyByteArray_AsString from bytearray that contains null characters

Question

I've got some buffer of data, a PyByteArray, that I want to extract the char * from. I want to then take that char* and create a stringstream from it.

void CKKSwrapper::DeserializeContext(PyObject *obj) {
    // Is a PyByteArray object as determined by https://docs.python.org/3/c-api/bytearray.html#c.PyByteArray_Check
    PyObject_Print(obj, stdout, Py_PRINT_RAW);  // 1
    const char *s = PyByteArray_AsString(obj);  // 2
    std::cout << "Converted bytearray to string" << std::endl;  // 3 
    std::cout << s << std::endl;  // 4


    std::istringstream is(s);  // 5
    lbcrypto::CryptoContext<lbcrypto::DCRTPoly> new_cc; // 6
    Serial::Deserialize(new_cc, is, SerType::BINARY); // 7
}

The 2nd line const char *s = PyByteArray_AsString(obj); outputs a single character. From a previous question C++ c_str of std::string returns empty (the title isn't accurate), I know for a fact that the underlying data for the input object, PyObject *obj, has NULL characters in it.

Based on the API I don't see any immediate solutions. Does anyone have any suggestions?

Note: From the current codebase I need to go from

Server: (C++ -> Python) -> (sockets) -> Client: (Python -> C++)

so I cant really get around the PyObject.

score 1 · Accepted Answer · answered Apr 03 '21 at 22:16

If you read the description of all the PyByteArray functions, you will find a useful function called PyByteArray_Size. This gives you the size of the array.

std::string has many constructors. One of them constructs the std::string from a beginning and an ending iterator. A pointer meets those qualifications. So, you should be able to construct an entire std::string accordingly:

const char *ptr = PyByteArray_AsString(obj);
size_t n=PyByteArray_Size(obj);

std::string s{ptr, ptr+n};

std::cout << s << std::endl;

Workaround for PyByteArray_AsString from bytearray that contains null characters

1 Answers1