C++ DLL - Cross Thread Callbacks

Question

We are implementing a number of SDK's for our suite of hardware sensors.

Having successfully got a working C API for one of our sensors, we are now starting the arduous task of testing the SDK to ensure that we haven't introduced any fatal bugs, memory leaks or race conditions.

One of our engineers has reported that when developing the test application (a Qt Widgets application), an issue has occurred when hooking onto a callback which is executed from a separate thread within the DLL.

Here is the callback prototype:

#define API_CALL __cdecl

typedef struct {
    // msg fields...
    DWORD dwSize;
    // etc...
} MSG_CONTEXT, *PMSG_CONTEXT;

typedef void (API_CALL *SYS_MSG_CALLBACK)(const PMSG_CONTEXT, LPVOID);

#define API_FUNC __declspec(dllexport)
API_FUNC void SYS_RegisterCallback(SYS_MSG_CALLBACK pHandler, LPVOID pContext);

And it is attached in Qt as follows:

static void callbackHandler(const PMSG_CONTEXT msg, LPVOID context) {
    MainWindow *wnd = (MainWindow *)context;

    // *wnd is valid 
    // Call a function here
}

MainWindow::MainWindow(QWidget *parent) {
    SYS_RegisterCallback(callbackHandler, this); 
}

My question is this: is the callback executed on the thread which creates it or on the thread which executes it? In either case, I guess it need of some kind of synchronization method. Googling has resulted in a plethora of C# examples, which isn't really whats needed.

One thing under consideration is using the SendMessage or PostMessage functions rather than going down the callback route.

Could anyone offer any suggestions please at to how cross-thread safety could be achieved using callbacks? Or is the message pump route the way to go for a Windows-based SDK?

Answer 1

My question is this: is the callback executed on the thread which creates it or on the thread which executes it?

The thread which executes it.

Could anyone offer any suggestions please at to how cross-thread safety could be achieved using callbacks?

A simple std::queue<message> protected by a mutex is the simplest solution. Here is a good example.

Have the callback do nothing but enqueue the message, and consume it from your main thread.

Answer 2

Could anyone offer any suggestions please at to how cross-thread safety could be achieved using callbacks?

Yes - the standard way it's done in Qt: emit a signal from any thread. connect a slot/functor to it that executes in the context of an object living in your desired target thread. We can also do something equivalent that doesn't use signal/slots explicitly, but functions the same - after all, a slot/functor call is carried across threads in a QMetaCallEvent.

The code you show usually leads to undefined behavior since you're attempting to use the GUI object (MainWindow) from any thread other than the main thread, and it's not likely that the method you're calling is thread safe.

The proper way to do it would be to either make the MainWindow's method thread-safe, or invoke it in a thread-safe manner.

One possible approach is shown below; see this answer for for isSafe and postCall implementations.

static void callbackHandler(const PMSG_CONTEXT msg, LPVOID context) {
    auto wnd = reinterpret_cast<MainWindow*>(context);
    if (!isSafe(wnd))
      return postCall(wnd, [=]{ callbackHandler(msg, context); });

    // we're executing in wnd's thread context, any calls on wnd
    // will be thread-safe
    wnd->foo();
}

MainWindow::MainWindow(QWidget *parent) : QWidget(parent) {
    SYS_RegisterCallback(callbackHandler, this); 
}

See also this question about invoking functors across threads, and this question about invoking methods thread-safely.

Another solution would be to offer this functionality directly and have a SYS_RegisterThreadPumpCallback that would register a callback just as SYS_RegisterCallback does, but would assume that the receiving thread runs a message pump (as Qt's main thread does, and any QEventLoop-spinning thread does too). The callback would be delivered as a message to a hidden window whose wndProc then performs the actual call via a function pointer.

Yet another solution is SYS_RegisterThreadAPCCallback that would use QueueUserAPC to invoke the callback when the receiving thread is alertable. This performs somewhat better than messaging to a message pump, but can lead to trouble should the user code be alertable in unexpected places and thus unexpectedly reenter code that's not reentrant (note: reentrancy and thread safety are orthogonal concepts).

I personally would welcome an API that offered all three kinds of callbacks on Windows.