UNIX: Maximum number of threads in a program

Question

My c progam is not able to create more than 8 threads. It returns the error code EAGAIN(11). Which is for lack of resources available. Before posting this question I googled for its solution but could not get much out of that. Here are the detail I have found for my program and unix system.

My thread creation functions is :-

thread_initialise(File *CFG_FILE)
{
        int total_pthreads; //reads number of threads I want for the program from configuration file.
        int rc =0 ;
        for (i = 0; i < total_pthreads; i++) 
        {
            rc = pthread_create (&pthread_list[i], NULL, (fp)(begin_worker_pthread), NULL);
            if (rc !=0) printf("Thread creation Error Code: %d",rc);
        }
}

Memory consumed by my program while execution is: pmap -x <pid> = 1111844

Unix Version:uname -a = Linux 2.6.18-308.24.1.el5 #1 SMP Wed Nov 21 11:42:14 EST 2012 x86_64 x86_64 x86_64 GNU/Linux

Thread Max value in unix cat /proc/sys/kernel/threads-max = 81920

ulimit -u max user processes (-u) 16000

ulimit -a
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 40960
max locked memory       (kbytes, -l) 3000000
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 5857280
real-time priority              (-r) 0
stack size              (kbytes, -s) 512000
cpu time               (seconds, -t) unlimited
max user processes              (-u) 16000
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Please help how the maximum number of threads is calculated/ fixed by my system. I want to increase my threads to 32.

What about virtual memory uses? and stack size? ulimit -a will show it. — rakib_, Aug 27 '13 at 11:49
@CodeCodeCode have you tried increasing ulimit -u 32000 ? ie max user processes? — rakib_, Aug 27 '13 at 12:14
@MartinJames Is it too high? I dont know how much is required for my process? Is there any way we can find out that minimum Stack size for my executable ? — CodeCodeCode, Aug 27 '13 at 14:32

score 2 · Accepted Answer · edited May 23 '17 at 10:32

2

set ulimit -s 4000 from terminal. Now you can run more thread than before, but you will meet segmentation fault in some stage.

number of threads = total virtual memory / (stack size*1024*1024)

The number of threads per process can be increased by increasing total virtual memory or by decreasing stack size. But, decreasing stack size too much can lead to code failure due to stack overflow while max virtual memory is equals to the swap memory.

further information see this post clearly explained.

edited May 23 '17 at 10:32

Community

1
1

answered Aug 27 '13 at 12:31

sujin

2,813
2
21
33

It helped me by setting `ulimit -s 4000`. but I am not sure about how much minimum stack size can be there for my program. I am afraid if it may lead to program failure at some point after execution. Virtual memory is already `ulimit -v unlimited` – CodeCodeCode Aug 27 '13 at 12:53
max virtual memory is equals to the swap memory. setup swap memeory for your os this may help. I'm not sure about this. – sujin Aug 27 '13 at 13:01
send the result of swapon -s – sujin Aug 27 '13 at 13:02
its says cmd not found: `swapon -s -bash: swapon: command not found` – CodeCodeCode Aug 27 '13 at 13:49
@CodeCodeCode which operating system you are using? – sujin Aug 27 '13 at 14:34
`number of threads = total virtual memory / (stack size*1024*1024)` My virtual memory is unlimted, so acc to your formula, the number of threds should be unlimted, atleast more than 8. – CodeCodeCode Aug 27 '13 at 14:34
@CodeCodeCode try this. cat /proc/swaps it will work under ubuntu – sujin Aug 27 '13 at 14:41

score 1 · Answer 2 · edited May 23 '17 at 10:32

After all above R&D I have found for my systems is that Maximum number of threads for program = Memory Size(RAM) / Stack Size. This calculation has worked for my system.

Though I have my Virtual Memory set as unlimited, I can not increase my number of threads than the above calculation.

In a sparate question I have asked for Minimum Stack Size for a program, so that the program never gets failed. Here is the link : UNIX: What should be Stack Size (ulimit -s) in UNIX?

UNIX: Maximum number of threads in a program

2 Answers2