0

I have written software which takes advantage of POSIX threads so that I can utilize shared memory within the process. My question is if I have a machine running Ubuntu with 4 processors and each processor has 16 cores. Is it more efficient to run 4 processes each with 16 threads or 1 process with 64 threads? Each processor has a dedicated 32gb of ram.

My main worry is that there will be a lot of memcopy happening behind the seen with 1 process.

In summary:
On a 4(16core) Proc Machine
1 process 64 threads?
4 Processes 16 Threads each?
If the process requires more than 32 gb of RAM(The amount dedicated to one Proc) does the answer differ?

Thanks for your help

RussS
  • 16,476
  • 1
  • 34
  • 62

1 Answers1

1

Depends on what your application does.

A thread in a single-threaded process runs faster then a thread in a multi-threaded process since the latter requires synchronization between threads in library functions like malloc(), fprintf(), etc.. Also, more threads in a multi-threaded process are likely to cause more lock contention slowing down each other. If threads don't need to communicate and don't share data they don't need to be in the same process.

In your case, you may get better parallelism with 4 processes with 16 threads rather then 1 process with 64 threads.

Maxim Egorushkin
  • 131,725
  • 17
  • 180
  • 271
  • The threads don't communicate but they do share data. There is an initial load of a large number of files and creation of shared structures which remain constant for the rest of the run. This can take up a large amount of ram which is why running multiple threads is preferable to multiple processes. Once the load is complete then the threads are created and act independently, there should be no synchronization or lock issues since none of the shared memory is changed. Does this change your opinion? – RussS Jan 27 '12 at 19:42
  • An option may be to load the data and fork into multiple processes instead of spawning threads. The child processes will still share all the data. This way it doesn't have to be compiled with threads support. However, it can make no noticeable difference. – Maxim Egorushkin Jan 27 '12 at 19:59
  • I was under the impression that fork() duplicates the memory in the parent process rather than sharing access to it? – RussS Jan 27 '12 at 20:09
  • From man fork: "Under Linux, fork() is implemented using copy-on-write pages, so the only penalty that it incurs is the time and memory required to duplicate the parent's page tables, and to create a unique task structure for the child." – Maxim Egorushkin Jan 27 '12 at 20:15
  • Thanks I didn't see that in the Notes section – RussS Jan 27 '12 at 20:19