44

I've been trying to familiarize myself with the std::thread library in C++11, and have arrived at a stumbling block.

Initially I come from a posix threads background, and was wondering how does one setup the stack size of the std::thread prior to construction, as I can't seem to find any references to performing such a task.

Using pthreads setting the stack size is done like this:

void* foo(void* arg);
.
.
.
.
pthread_attr_t attribute;
pthread_t thread;

pthread_attr_init(&attribute);
pthread_attr_setstacksize(&attribute,1024);
pthread_create(&thread,&attribute,foo,0);
pthread_join(thread,0);

Is there something similar when using std::thread?

I've been using the following reference:

http://en.cppreference.com/w/cpp/thread

Zamfir Kerlukson
  • 1,046
  • 1
  • 10
  • 20
  • you could get native handler and set stack size? http://en.cppreference.com/w/cpp/thread/thread/native_handle – billz Dec 14 '12 at 02:20
  • 9
    @billz Once the thread has been created it is too late. – Zamfir Kerlukson Dec 14 '12 at 02:21
  • Why do you want to do this? – Billy ONeal Dec 14 '12 at 02:28
  • 2
    @Billy: There are many scenarios where one may want to create many threads for a thread pool and assume that all the data/memory the thread will require will external hence the stack can be set small - essentially enough to house any calls or code nothing else. – Zamfir Kerlukson Dec 14 '12 at 02:37
  • 2
    @BillyONeal i'm working on a program that needs a lot worker threads, these threads doesn't need a big stack size (running them at a 500 kb stack/worker thread), the std::thread default stack size is 8 megabytes/thread. if i want 10 worker threads, that's either 80 megabytes of ram using std::thread, or 5 megabytes of ram using pthreads.. – hanshenrik Aug 31 '18 at 14:22
  • @hanshenrik: Not really. 8MB (on Linux) and 1MB (on Windows) is the default stack size *reservation* -- it'll only generally consume physical RAM if pages are touched. – Billy ONeal Sep 09 '18 at 19:26
  • @BillyONeal so you're saying a thread that doesn't do anything, just `void threadEntry(){for(;;){this_thread::sleep_for(seconds(1));}}` but have a stack size of 0.5MB use just as much memory as a thread doing the same with a 8MB stack size? – hanshenrik Sep 09 '18 at 20:17
  • 1
    @BillyONeal damn, seems you're right, i wrote some [code](https://gist.github.com/divinity76/38ab577e546c9828cd58ea2544c87801) to test it, then installed a debian 9 amd64 minimal with kernel 4.9.110 , g++ 6.3.0, and 152 MB ram and 0 swap (which seems to be nearly the minimum debian will boot on without tweaking the `initial ramdisk` - somewhere between 140 and 152MB ram) on VMWare, and both mode 1 (std::thread with 8MB stack) and mode 2 (pthread with 512 kb stack) are able to run *exactly* 700 threads before failing! cool - during boot it needs ~152 MB but after boot its 65MB ram, unfortunate – hanshenrik Sep 09 '18 at 22:05

6 Answers6

27

Initially I come from a posix threads background, and was wondering how does one setup the stack size of the std::thread prior to construction, as I can't seem to find any references to performing such a task.

You can't. std::thread doesn't support this because std::thread is standardized, and C++ does not require that a machine even has a stack, much less a fixed-size one.

pthreads are more restrictive in terms of the hardware that they support, and it assumes that there is some fixed stack size per thread. (So you can configure this)

Billy ONeal
  • 104,103
  • 58
  • 317
  • 552
  • 8
    +1 but that is a very disappointing and short-sighted result. I'd have expected a trait or stack allocator trait mechanism at the very least. – Zamfir Kerlukson Dec 14 '12 at 02:39
  • 11
    @ZamfirKerlukson: The language designers did think about this. You get the native handle from the Thread object. Then manipulate it with appropriate native function calls for your platform. But the reason they made it hard is that doing it is usually a mistake and when actually required is very/very rare. – Martin York Dec 14 '12 at 04:20
  • 6
    I don't see how to change the stack size using the native handle. Any hints? Moreover, the Mac OSX default for a pthread stack size is 512kB. This is quite small. – Leonardo de Moura Dec 03 '13 at 06:00
  • 2
    As far as I am aware you can't change the stack size for a thread after it has started. – Billy ONeal Nov 01 '16 at 18:23
  • I like so generic software that it has 0 flexibility. I wonder if there exists an implementation that has no stack? – Martin Kosicky Apr 30 '20 at 14:39
19

As Loki Astari already said, it is extremely rare to actually need a non-default stack-size and usually either a mistake or the result of bad coding.

  • If you feel like the default stack size is too big for your needs and want to reduce it, just forget about it. Every modern OS now uses virtual memory / on-demand commit, which means that memory is only reserved, not actually allocated, until you access the pages. Reducing the stack size will not reduce your actual memory footprint.

  • Due to this very behaviour, OSes can afford to set the default stack size to very big values. E.g. on a vanilla Debian this is 8MB (ulimit -s) which should be enough for every need. If you still manage to hit that limit, my first idea would be that your code is wrong, so you should first and foremost review it and move things to the heap, transform recursive functions into loops, etc.

  • If despite all of this you really really need to change the stack size (i.e. increase it, since reducing it is useless), on POSIX you can always use setrlimit at the start of your program to increase the default stack size. Sure this will affect all threads, but only the ones who need it will actually use the additional memory.

  • Last but not least, in all fairness I can see a corner case where reducing the stack size would make sense: if you have tons of threads on a 32 bits system, they could eat up your virtual address space (again, not the actual memory consumption) up to the point that you don't have enough address space available for the heap. Again, setrlimit is your friend here even though I'd advise to move to a 64 bits system to benefit from the larger virtual address space (and if your program is that big anyway, you'll probably benefit from the additional RAM too).

syam
  • 14,701
  • 3
  • 41
  • 65
  • 4
    I tried `setrlimit` on Linux and Cygwin. On Cygwin, it always fails. On Linux, it only changes the size of the main thread. The `ulimit -s` solution works on Linux. On OSX, it only affects the size of the main thread. The Mac OS X default for a pthread stack size is 512kB. This is quite small. It is a pity we can't change the default stack size using `std::thread`. – Leonardo de Moura Dec 03 '13 at 05:30
  • 4
    640 kb used to be enough for everybody – user1095108 Oct 19 '16 at 10:36
  • 4
    ", it is extremely rare to actually need a non-default stack-size". Speak for yourself. I work with large matrix operations and often the default stack size is not enough. –  Oct 13 '17 at 16:30
  • 2
    On microcontrollers you most likely want to be able to set the stack size. Just another agument to support the preconception that C++ is not good for microcontrollers. Sad. – Sebastian Mar 06 '18 at 17:41
  • @Sebastian I agree that being able to control stack sizes on a microcontroller would be helpful, but I feel it's more common to be using an RTOS in those environments in which case you'd be using that RTOS's API for thread manipulation. C++ is still viable there, albeit with some features being less useful. – Rich von Lehe Aug 30 '18 at 14:59
  • 1
    Windows has a 1MB default stack. Clang has to increase the stack size to have enough space to instantiate recursive templates. Are they writing bad code? – Alex Reinking Mar 31 '21 at 20:58
18

I have also been investigating this issue. For some applications, the default stack size is not adequate. Examples: the program does deep recursion dependent on the specific problem it is solving; the program needs to create many threads and memory consumption is an issue.

Here is a summary of (partial) solutions / workarounds I found:

  • g++ supports a -fsplit-stack option on Linux. See for more information about Split stacks. Here is summary from their website:

The goal of split stacks is to permit a discontiguous stack which is grown automatically as needed. This means that you can run multiple threads, each starting with a small stack, and have the stack grow and shrink as required by the program.

Remark: -fsplit-stack only worked for me after I started using the gold linker. It seems clang++ will also support this flag. The version I tried (clang++ 3.3) crashed when trying to compile my application using the flag -fsplit-stack.

  • On Linux, set the stack size by executing ulimit -s <size> before starting your application. size is the stack size in Kbs. Remark: the command unlimit -s unlimited did not affect the size of threads created with std::thread. When I used ulimit -s unlimited, the main thread could grow, but the threads created with std::thread had the default size.

  • On Windows using Visual Studio, we can use use the linker /STACK parameter or /STACKSIZE in the module definition file, this is the default size for all created threads. See this link for more information. We can also modify this parameter in any executable using the command line tool EDITBIN.

  • On Windows using mingw g++, we can use the option -Wl,--stack,<size>. For some reason, when using cygwin g++, this flag only affects the size of the main thread.

Approaches that did not work for me:

  • ulimit -s <size> on OSX. It affects only the size of the main thread. Moreover, the Mac OSX default for a pthread stack size is 512kB.

  • setrlimit only affects the size of the main thread on Linux and OSX. On cygwin, it never worked for me, it seems it always returns an error.

For OSX, the only alternative seems to use boost::thread instead of std::thread, but this is not nice if we want to stick with the standard. I hope g++ and clang++ will also support -fsplit-stack on OSX in the future.

Leonardo de Moura
  • 21,065
  • 2
  • 47
  • 53
16

I found this in Scott Meyers book Overview of the New C++(C++0x), as it's quite long I can't post it as a comment, is this helpful?

There is also a standard API for getting at the platform-specific handles behind threads, mutexes, condition variables, etc.. These handles are assumed to be the mechanism for setting thread priorities, setting stack sizes, etc. (Regarding setting stack sizes, Anthony Williams notes: "Of those OSs that support setting the stack size, they all do it differently. If you're coding for a specify platform (such that use of the native_handle would be OK), then you could use that platform's facilities to switch stacks. e.g. on POSIX you could use makecontext and swapcontext along with explicit allocation of a stack, and on Windows you could use Fibers. You could then use the platform-specific facilities (e.g. Linker flags) to set the default stack size to something really tiny, and then switch stacks to something bigger where necessary.“)

billz
  • 44,644
  • 9
  • 83
  • 100
15

Was looking for the answer to this myself just now.

It appears that while std::thread does not support this, boost::thread does.

In particular, you can use boost::thread::attributes to accomplish this:

boost::thread::attributes attrs;
attrs.set_stack_size(4096*10);
boost::thread myThread(attrs, fooFunction, 42);
Innot Kauker
  • 1,333
  • 1
  • 18
  • 30
Mark
  • 3,806
  • 3
  • 21
  • 32
1

You can do some modifications like this if you don't want to include a big library.

It is still dependend C++ compiler STL library. (Clang / MSVC now)

HackingSTL Library

std::thread thread = std::stacking_thread(65536, []{
    printf("Hello, world!\n"); 
});
IndieGameDev
  • 2,905
  • 3
  • 16
  • 29
metarutaiga
  • 11
  • 1