Using a c++ std::mutex declared inside a python object

Question

Short question : How can i Initialize a std::mutex in a structure after using malloc to get the corresponding memory?

More detail: i'm trying to make a python module with a background thread

I create and run the thread without any problem, but if I try to use a mutex stored in the Python object, it crash.

If I define the mutex at the head of my cpp file, it works. But I would prefer to store it within the Device structure

In my PyDevice.h

#include <Python.h>
#include "structmember.h"

typedef struct sSharedData
{
    bool mStop;
} sSharedData;

typedef struct sDevice
{
    PyObject_HEAD
    std::thread mDataThread;
    std::mutex mDataThreadMutex;
    sSharedData mDataThreadSharedData;
} sLeddarDevice;


PyObject *StartDataThread( sLeddarDevice *self, PyObject *args );
PyObject *StopDataThread( sLeddarDevice *self, PyObject *args );

static PyMethodDef Device_methods[] =
{
    { "StartDataThread", ( PyCFunction )StartDataThread, METH_NOARGS, "Start the thread." },
    { "StopDataThread", ( PyCFunction )StopDataThread, METH_NOARGS, "Stop the thread." },
    { NULL }  //Sentinel
};

static PyMemberDef Device_members[] =
{
    { NULL }  //Sentinel
};

static PyTypeObject LeddarDeviceType =
{
    PyObject_HEAD_INIT( NULL )
    0,                              //ob_size
    "LeddarPy.Device",              //tp_name
    sizeof( sDevice ),        //tp_basicsize
    0,                              //tp_itemsize
    ( destructor )Device_dealloc,   //tp_dealloc
    0,                              //tp_print
    0,                              //tp_getattr
    0,                              //tp_setattr
    0,                              //tp_compare
    0,                              //tp_repr
    0,                              //tp_as_number
    0,                              //tp_as_sequence
    0,                              //tp_as_mapping
    0,                              //tp_hash
    0,                              //tp_call
    0,                              //tp_str
    0,                              //tp_getattro
    0,                              //tp_setattro
    0,                              //tp_as_buffer
    Py_TPFLAGS_DEFAULT,             //tp_flags
    "Device object.",               // tp_doc
    0,                              //tp_traverse
    0,                              //tp_clear
    0,                              //tp_richcompare
    0,                              //tp_weaklistoffset
    0,                              //tp_iter
    0,                              //tp_iternext
    Device_methods,                 //tp_methods
    Device_members,                 //tp_members
    0,                              //tp_getset
    0,                              //tp_base
    0,                              //tp_dict
    0,                              //tp_descr_get
    0,                              //tp_descr_set
    0,                              //tp_dictoffset
    0,                              //tp_init
    0,                              //tp_alloc
    Device_new,                     //tp_new
};

And in my PyDevice.cpp

#include "LeddarPyDevice.h"
//Constructor
PyObject *Device_new( PyTypeObject *type, PyObject *args, PyObject *kwds )
{
    sLeddarDevice *self;

    self = ( sLeddarDevice * )type->tp_alloc( type, 0 );

    if( self != nullptr )
    {
        self->mDataThreadSharedData.mStop = false;
    }

    return ( PyObject * )self;
}

//Destructor
void Device_dealloc( sLeddarDevice *self )
{
    DebugTrace( "Destructing device." );
    Py_TYPE( self )->tp_free( ( PyObject * )self );
}

PyObject *StartDataThread( sLeddarDevice *self, PyObject *args )
{
    DebugTrace( "Starting thread" );
    self->mDataThreadMutex.lock();
    self->mDataThreadSharedData.mStop = false;
    self->mDataThreadMutex.unlock();
    self->mDataThread = std::thread( DataThread, self );

    Py_RETURN_TRUE;
}

It crashes whenever I try to use self->mDataThreadMutex.lock().

Im not sure if the mutex is initialized correctly, i'm more used to pthread mutex where you need to initiliaze it manually.

Answer 1

Python is written in C and as such it won't run C++ constructors/destructors for you. And without them std::mutex is just a meaningless struct which will crash when used. What you need is placement new somewhere (after allocating self?):

self = ( sLeddarDevice * )type->tp_alloc( type, 0 );
new (self) sLeddarDevice {};

The second line calls the constructor of sLeddarDevice on self without allocating memory. That constructor will call constructors of each member as well.

You also have to manually call destructors when done: self->~sLeddarDevice();

You should always do that when combining C++ with Python. Otherwise in best case scenario you will leak memory. In worse case encounter undefined behaviour with random crashes.