19

I am beginner in this area.

I have studied fork(), vfork(), clone() and pthreads.

I have noticed that pthread_create() will create a thread, which is less overhead than creating a new process with fork(). Additionally the thread will share file descriptors, memory, etc with parent process.

But when is fork() and clone() better than pthreads? Can you please explain it to me by giving real world example?

Thanks in Advance.

phs
  • 10,687
  • 4
  • 58
  • 84
Brijesh Valera
  • 1,085
  • 2
  • 9
  • 30
  • 4
    You are comparing 2 different things. None is better. Its just, which is applicable where. – Vinayak Garg Jul 26 '12 at 05:38
  • Ohkk.. Gottaa We want separate execution then fork() is better and when we want some execution on same address space by different thread then pthread is cool. is it so? – Brijesh Valera Jul 26 '12 at 05:46
  • 2
    `pthread_create()` calls `clone()` – Zaffy Dec 15 '12 at 15:08
  • 2
    `pthread_create()` is _not_ less overhead than `fork()`. In fact it is _more_. Both call `clone` syscall internally, but pthread user space library needs to update a lot of threads tracking structures in userspace additionally to creating new kernel task. – smokku Oct 05 '15 at 22:55

4 Answers4

37

clone(2) is a Linux specific syscall mostly used to implement threads (in particular, it is used for pthread_create). With various arguments, clone can also have a fork(2)-like behavior. Very few people directly use clone, using the pthread library is more portable. You probably need to directly call clone(2) syscall only if you are implementing your own thread library - a competitor to Posix-threads - and this is very tricky (in particular because locking may require using futex(2) syscall in machine-tuned assembly-coded routines, see futex(7)). You don't want to directly use clone or futex because the pthreads are much simpler to use.

(The other pthread functions require some book-keeping to be done internally in libpthread.so after a clone during a pthread_create)

As Jonathon answered, processes have their own address space and file descriptor set. And a process can execute a new executable program with the execve syscall which basically initialize the address space, the stack and registers for starting a new program (but the file descriptors may be kept, unless using close-on-exec flag, e.g. thru O_CLOEXEC for open).

On Unix-like systems, all processes (except the very first process, usuallyinit, of pid 1) are created by fork (or variants like vfork; you could, but don't want to, use clone in such way as it behaves like fork).

(technically, on Linux, there are some few weird exceptions which you can ignore, notably kernel processes or threads and some rare kernel-initiated starting of processes like /sbin/hotplug ....)

The fork and execve syscalls are central to Unix process creation (with waitpid and related syscalls).

A multi-threaded process has several threads (usually created by pthread_create) all sharing the same address space and file descriptors. You use threads when you want to work in parallel on the same data within the same address space, but then you should care about synchronization and locking. Read a pthread tutorial for more.

I suggest you to read a good Unix programming book like Advanced Unix Programming and/or the (freely available) Advanced Linux Programming

Community
  • 1
  • 1
Basile Starynkevitch
  • 223,805
  • 18
  • 296
  • 547
  • So if I was creating a HTTP web server that has to respond to multiple incoming requests, would it be preferable to use pthreads instead? – RoadRunner Apr 02 '18 at 09:27
  • There is no general rule. However, web servers are indeed a good case for a multithreaded approach (but there are also multi-process webservers). Look also into [libonion](https://www.coralbits.com/libonion/), an HTTP server library – Basile Starynkevitch Apr 02 '18 at 16:57
14

The strength and weakness of fork (and company) is that they create a new process that's a clone of the existing process.

This is a weakness because, as you pointed out, creating a new process has a fair amount of overhead. It also means communication between the processes has to be done via some "approved" channel (pipes, sockets, files, shared-memory region, etc.)

This is a strength because it provides (much) greater isolation between the parent and the child. If, for example, a child process crashes, you can kill it and start another fairly easily. By contrast, if a child thread dies, killing it is problematic at best -- it's impossible to be certain what resources that thread held exclusively, so you can't clean up after it. Likewise, since all the threads in a process share a common address space, one thread that ran into a problem could overwrite data being used by all the other threads, so just killing that one thread wouldn't necessarily be enough to clean up the mess.

In other words, using threads is a little bit of a gamble. As long as your code is all clean, you can gain some efficiency by using multiple threads in a single process. Using multiple processes adds a bit of overhead, but can make your code quite a bit more robust, because it limits the damage a single problem can cause, and makes it much easy to shut down and replace a process if it does run into a major problem.

As far as concrete examples go, Apache might be a pretty good one. It will use multiple threads per process, but to limit the damage in case of problems (among other things), it limits the number of threads per process, and can/will spawn several separate processes running concurrently as well. On a decent server you might have, for example, 8 processes with 8 threads each. The large number of threads helps it service a large number of clients in a mostly I/O bound task, and breaking it up into processes means if a problem does arise, it doesn't suddenly become completely un-responsive, and can shut down and restart a process without losing a lot.

Jerry Coffin
  • 476,176
  • 80
  • 629
  • 1,111
  • 2
    It is a myth that creating a process is more overhead than creating a thread. On Linux both are kernel tasks internally, created with the same overhead (thanks to copy-on-write "magic"). It is also a myth that communicating using shared resources is more efficient than using messaging IPC - especially on modern multi-core machines, where communicating threads may reside on different CPU dies. – smokku Oct 05 '15 at 23:00
  • 2
    @smoku: No, it is not a myth. Even with COW, creating a new process requires duplicating the existing process' page table, where creating a thread just re-uses the existing page table. Yes, it reduces the amount of memory that needs to be copied by a few orders of magnitude, but on a process that's using a lot of memory, the page table can still easily be a few megabytes. – Jerry Coffin Oct 06 '15 at 00:13
  • Copying even a few megabytes of memory is a non-issue today. It's not 1980s anymore. – smokku Oct 06 '15 at 08:02
  • 4
    @smoku: Nobody said it was an issue. You, however, said that it was a myth that creating a process had more overhead than creating a thread. If you want to change your position to one of claiming that the overhead is often entirely reasonable, I quite agree--but claiming that copying megabytes of data imposes no overhead compared to *not* copying those megabytes of data at all is simply an outright falsehood. – Jerry Coffin Oct 06 '15 at 13:03
  • Ok. I can agree with that reasoning. :-) Still, how you are selling it is still so '80s thinking and keeps perpetuating the myth that choosing threads over full processes is a no-brainer, because threads are soooo lightweight. Also once you add all the pthread bookkeeping over a cloned task, I'm not sure it's still more overhead to fork a process than pthread_create(). :-) – smokku Oct 06 '15 at 16:47
  • 1
    @smoku: I'm not selling anything here. I will say that in my experience, you can reduce overhead (both for spawning and communication) by using threads rather than separate tasks all the time. Essentially everybody developing web servers (for only one obvious example) seems to agree. At the same time, I think I gave a pretty balanced view--that this approach has shortcomings as well. – Jerry Coffin Oct 06 '15 at 17:19
6

These are totally different things. fork() creates a new process. pthread_create() creates a new thread, which runs under the context of the same process.

Thread share the same virtual address space, memory (for good or for bad), set of open file descriptors, among other things.

Processes are (essentially) totally separate from each other and cannot modify each other.

You should read this question:

As for an example, if I am your shell (eg. bash), when you enter a command like ls, I am going to fork() a new process, and then exec() the ls executable. (And then I wait() on the child process, but that's getting out of scope.) This happens in an entire different address space, and if ls blows up, I don't care, because I am still executing in my own process.

On the other hand, say I am a math program, and I have been asked to multiply two 100x100 matrices. We know that matrix multiplication is an Embarrassingly Parallel problem. So, I have the matrices in memory. I spawn of N threads, who each operate on the same source matrices, putting their results in the appropriate location in the result matrix. Remember, these operate in the context of the same process, so I need to make sure they are not stamping on each other's data. If N is 8 and I have an eight-core CPU, I can effectively calculate each part of the matrix simultaneously.

Community
  • 1
  • 1
Jonathon Reinhart
  • 132,704
  • 33
  • 254
  • 328
  • so fork() is only used when we want that execution must be going on separate space which will not affect the calling process. is it the only thing? – Brijesh Valera Jul 26 '12 at 05:41
  • 1
    No, `fork` (with its variant `vfork`) is *the only* way to create a process (unless you use the Linux-specific `clone` syscall in such ways that it exactly duplicates the behavior of `fork`). All application processes are created by `fork`except the very first process `init` (and some weird Linux-specific exceptions). – Basile Starynkevitch Jul 26 '12 at 10:36
0

Process creation mechanism on unix using fork() (and family) is very efficient. Morever , most unix system doesnot support kernel level threads i.e thread is not entity recognized by kernel. Hence thread on such system cannot get benefit of CPU scheduling at kernel level. pthread library does that which is not kerenl rather some process itself. Also on such system pthreads are implemented using vfork() and as light weight process only. So using threading has no point except portability on such system.

As per my understanding Sun-solaris and windows has kernel level thread and linux family doesn't support kernel threads.

with processes pipes and unix doamin sockets are very efficient IPC without synchronization issues. I hope it clears why and when thread should be used practically.

yogesh
  • 1