12

Is it possible by any means to change the limit on the number of pthreads a process can create ? Currently on my linux system I can create around 380 threads but I want to increase that to say as long as memory is available.

Sukanto
  • 992
  • 3
  • 11
  • 21
  • 2
    Interesting... have you considered some of the implications of such system design? Single Point Of Failure (SPOF) comes to mind. – jldupont Dec 22 '09 at 15:53
  • I'm no expert, but I would bet that having much more than 380 threads will overwhelm your processor with context switches. Context switching for a thread is less than for a separate process, but you still have to save the program counter and register file and perhaps some other stuff I'm forgetting. How many threads do you think you need? I'd suggest profiling your application to determine the thread overhead for 10 threads versus 100 threads and then try to extrapolate to your target number of threads. You may find that it ends up not being worth the effort. Or perhaps I'm just wrong... – A. Levy Dec 22 '09 at 16:02
  • I won't be using it practically anywhere ... but I need it for some experiments ? – Sukanto Dec 22 '09 at 16:16
  • 1
    setrlimit() man pages (http://linux.die.net/man/2/setrlimit) say : "RLIMIT_NPROC: The maximum number of threads that can be created for the real user ID of the calling process." On calling `getrlimit(RLIMIT_NPROC, &rlim);` I get the following resource Limits: `Current=4294967295 Max=4294967295` – Sukanto Dec 22 '09 at 16:29
  • RLIMIT_NPROC gives the maximum number of threads per user. – Sukanto Dec 22 '09 at 16:37
  • Have you managed to resolve this issue ? – Demiurg Jul 21 '11 at 20:52

4 Answers4

6

reduce user's stack size 'ulimit -s 1024'; 

default: 8MB 
reduced: 1MB

to increase number of threads.

set the stack size: pthread_attr_setstacksize(1024)

osgx
  • 90,338
  • 53
  • 357
  • 513
saju
  • 61
  • 1
  • 1
3
cat /proc/sys/kernel/threads-max

may work in Linux but not other UNIX systems. I thought the right way is

Maximum number of threads per process - sysconf(_SC_THREAD_THREADS_MAX) failing

which works on some UNIX systems (like HPUX) but not on Solaris or Linux...

Community
  • 1
  • 1
Reza Toghraee
  • 1,603
  • 1
  • 14
  • 21
2

Your problem is that you have not called pthread_detach on the threads in question. This tells pthread that the resources associated with each thread will be released when the thread terminates. You have to call either pthread_join or pthread_release on all threads to release thread resources. That means that you also have to call pthread_detach in your pthread_join cancellation handlers or leak.

user259759
  • 21
  • 2
1

Look at this:

Maximum number of threads per process in Linux?

And take a look at this as it might pertain to your question:

Serve one client with each server thread

Community
  • 1
  • 1
  • `cat /proc/sys/kernel/threads-max` shows `16384`. But I cannot create more than 380 pthreads. I even tried putting a blank function as body of the threads. – Sukanto Dec 22 '09 at 16:34
  • after 380 threads I get `EAGAIN` as return value of `pthread_create()`, which according to `pthread_create()` man page is "The system lacked the necessary resources to create another thread, or the system-imposed limit on the total number of threads in a process {PTHREAD_THREADS_MAX} would be exceeded." – Sukanto Dec 23 '09 at 08:29
  • 1
    Are you testing 32-bit application? If so have you checked that when you create a 381 thread you have enough space for stack of the new thread? –  Dec 23 '09 at 10:16
  • Or there might be a limit on virtual memory size for a process. You can see limits running `ulimit -a` –  Dec 23 '09 at 11:01
  • virtual memory size limit is `unlimited` – Sukanto Dec 24 '09 at 11:17