Is it safe to mix boost::thread with C++11 std::mutex?

Question

Is it safe to use std::mutex and its kin in a program that starts its threads via boost?

(Using std::thread is not an option for me (I think), as the application needs a lot of stack space, and on some platforms requires overriding the default stack size upon creation.)

Yes, it's the safe way. `std::mutex` is just a layer between your program and OS API calls. Also `boost::thread`. — Mister_Jesus, Feb 15 '19 at 21:57
Why not use boost::mutex if you are worried? if you use the mutexes from boost itself you get the benefit of all its features that std::mutex don't have. Like the `UpgradeLockable` concept — Mellester, Feb 15 '19 at 22:02
@MrBin That doesn't sound safe to me at all - not after reading the answers to [this related question](https://stackoverflow.com/questions/29710001/is-it-okay-to-use-stdthis-thread-functions-from-boostthreads/29712820#29712820). — Christoph Lipka, Feb 15 '19 at 22:34
@Mellester I'm trying to reduce the "library footprint" of a core module, so that it can be used outside its current ecosystem with as few non-standard libraries as possible. Clearing the core module from boost::thread would be a huge win in that respect, most notably since it would eliminate the requirement to link to an extra lib (as opposed to many other boost components that are header-only). — Christoph Lipka, Feb 15 '19 at 22:41
You would have to really to stick to very primitive objects for locks. you cant really be sure std::lock(m1,m2) doesnt use somthing like std::thread::sleep_until because those are undefined in boos::thread. — Mellester, Feb 16 '19 at 00:43
Good idea to try to reduce the dependence on Boost. But you also shouldn't set the stack size from within the application, there is most likely a linker option for that for your specific platform ([example](https://learn.microsoft.com/en-us/cpp/build/reference/f-set-stack-size?view=vs-2017), [example](https://stackoverflow.com/questions/2275550/change-stack-size-for-a-c-application-in-linux-during-compilation-with-gnu-com)). Then you can switch to `std::thread` and not worry about mixing libraries. — rustyx, Feb 16 '19 at 08:59
Are you sure the application needs a lot of stack space? I cannot see an application which fills 8MB of stack space. — fiorentinoing, Feb 16 '19 at 20:28
@rustyx In an ideal world we would rely on compiler or system settings to control the stack size; but unfortunately on one of our target platforms the stack size for additional threads can _only_ be controlled from the running application (looking at you, OS X). — Christoph Lipka, Feb 16 '19 at 20:32
@fiorentinoing On OS X we're talking about a limit of 512kiB per thread, not 8MiB. And we did see some real-life cases where even 4MiB were insufficient. (The software has been changed since then, and I hope we can do with 512kiB now, but tests are still pending.) — Christoph Lipka, Feb 16 '19 at 20:43
@ChristophLipka I'd give a try to valgrind (massif plugin) to understand where stack is used and change to heap with RAII. Did you this? — fiorentinoing, Feb 16 '19 at 21:21
@fiorentinoing I'm a Windows jockey; valgrind is beyond my paygrade ;) But yes, getting rid of more stack-hogging structures is an option I may be looking into. But it probably won't be as trivial as just allocating it on the heap via a simple RAII wrapper, due to performance constraints, so I'm still trying to avoid opening that can of worms. — Christoph Lipka, Feb 16 '19 at 23:10
First test results are in now, and they look very promising: In a case that used to require a whopping 2 MiB of stack, we now get away with 16 KiB. So unless something unexpected crops up in other tests, the default stack size should be enough for everybody, even on Mac OS X. — Christoph Lipka, Feb 17 '19 at 01:48

score 1 · Answer 1 · answered Nov 25 '19 at 10:39

1

Yes, you can use std::mutex in threads created with boost::thread.

answered Nov 25 '19 at 10:39

Anthony Williams

66,628
14
133
155

Maxim Egorushkin · Answer 2 · 2019-02-20T10:45:10.303

0

the application needs a lot of stack space, and on some platforms requires overriding the default stack size upon creation

std::thread uses pthread_create to create threads on platforms that support POSIX. Another option for you is to override pthread_create, set the stack size and call the original pthread_create.

A working Linux example (I don't have access to MacOS to try it there):

#include <cstdio>
#include <thread>
#include <dlfcn.h>

namespace {

size_t const STACK_SIZE = 8 * 1024 * 1024;

int pthread_create_override(pthread_t* thread, pthread_attr_t const* attr, void*(*start_routine)(void*), void* arg) noexcept {
    std::printf("%s\n", __PRETTY_FUNCTION__);

    pthread_attr_t attr2;
    if(attr)
        attr2 = *attr;
    else
        if(pthread_attr_init(&attr2))
            std::abort();

    size_t stacksize = 0;
    pthread_attr_getstacksize(&attr2, &stacksize);
    if(stacksize < STACK_SIZE) {
        if(pthread_attr_setstacksize(&attr2, STACK_SIZE))
            std::abort();
    }

    static auto const real_pthread_create = reinterpret_cast<decltype(&pthread_create)>(::dlsym(RTLD_NEXT, "pthread_create"));
    int rc = real_pthread_create(thread, &attr2, start_routine, arg);

    if(!attr)
        pthread_attr_destroy(&attr2);

    return rc;
}

} // namespace

extern "C" {

int pthread_create(pthread_t* thread, pthread_attr_t const* attr, void*(*start_routine)(void*), void* arg) {
    return pthread_create_override(thread, attr, start_routine, arg);
}

} // namespace

int main() {
    std::thread t([]() { std::printf("%s\n", __PRETTY_FUNCTION__); });
    t.join();
}

Outputs:

int {anonymous}::pthread_create_override(pthread_t*, const pthread_attr_t*, void* (*)(void*), void*)
main()::<lambda()>

edited Feb 20 '19 at 10:45

answered Feb 17 '19 at 19:42

Maxim Egorushkin

131,725
17
180
271

Both the `ulimit` and `setrlimit` approaches are known to affect only the main thread on Mac OS X. – Christoph Lipka Feb 18 '19 at 20:47
Whether `std::thread` calls `pthread_create` - let alone whether this call is implemented in such a way that it can simply be overridden by providing my own `pthread_create` - is most certainly implementation-defined, and relying on it to remain implemented in this manner is calling for trouble. – Christoph Lipka Feb 18 '19 at 20:53
@ChristophLipka Can you verify on your Mac OS system whether overriding `pthread_create` solves your problem? Your standard library most likely uses the standard `pthread_create`, because `std::thread` was designed as a Pthread wrapper and Mac OS X does provide this API. – Maxim Egorushkin Feb 18 '19 at 23:27
can you provide an authoritative source for your claim that `std::thread` was "designed as a Pthread wrapper"? Because i'm pretty sure the standard was specifically NOT designed as a wrapper for any _particular_ threads implementation. (And no, I cannot verify on "my Mac OS system", because I don't have such a system at my disposal myself; let alone that Mac OS is not our only target platform.) – Christoph Lipka Feb 18 '19 at 23:42
@ChristophLipka That is a common knowledge that C++ threading mimics Pthread API, and that is obvious to anyone familiar with both APIs. – Maxim Egorushkin Feb 20 '19 at 10:46
If `std::thread` mimics the Pthread API, then where are the thread attributes? I don't see them anywhere. – Christoph Lipka Feb 21 '19 at 22:00
1

@ChristophLipka http://pubs.opengroup.org/onlinepubs/9699919799/help/codes.html#TSS : _Thread Stack Size Attribute The functionality described is optional. The functionality described is also an extension to the ISO C standard._ – Maxim Egorushkin Feb 21 '19 at 22:13

Maxim Egorushkin · Answer 3 · 2019-02-22T10:53:05.560

The way you start threads (pthread_create, boost::thread, std::thread) is orthogonal to synchronisation primitives you use (std vs boost vs Intel TBB vs libatomic vs etc..).

Otherwise, you wouldn't be able to mix and match libraries that use these different APIs in one application.

Low-level synchronisation primitives such as atomics, mutexes and condition variables, can be placed in shared memory and used by different processes without creating any extra threads explicitly at all. As well as the main application thread is created by the OS kernel for you without using any of the user space APIs.

Is it safe to mix boost::thread with C++11 std::mutex?

3 Answers3