19
from multiprocessing import Process
# c is a container
p = Process(target = f, args = (c,))
p.start()

I assume a deep copy of c is passed to function f because shallow copy would make no sense in the case of a new process (the new process doesn't have access to the data from the calling process).

But how is this deep copy defined? There is a whole set of notes in the copy.deepcopy() documentation, do all these notes apply here as well? The multiprocessing documentation says nothing...

max
  • 49,282
  • 56
  • 208
  • 355

1 Answers1

19

When you create a Process instance, under the hood Python issues a fork(). This creates a child process whose memory space is an exact copy of its parent -- so everything existing at the time of the fork is copied.

On Linux this is made efficient through "copy-on-write". From the fork man page:

fork() creates a child process that differs from the parent process only in its PID and PPID, and in the fact that resource utilizations are set to 0. File locks and pending signals are not inherited.

Under Linux, fork() is implemented using copy-on-write pages, so the only penalty that it incurs is the time and memory required to duplicate the parent's page tables, and to create a unique task structure for the child.

samplebias
  • 37,113
  • 6
  • 107
  • 103
  • 1
    +1. Note that Windows uses a different mechanism than `fork()`, but the effects are similar. – Sven Marnach May 12 '11 at 19:33
  • This is correct, but one has to mention that it isn't true that nothing gets copied. Python's garbage collector needs to maintain ref counts. The memory overhead for this is negligible though (I think 4kb per each process). –  Sep 20 '15 at 19:54
  • Old information above: From the [python docs](https://docs.python.org/3/library/multiprocessing.html?highlight=multiprocessing#contexts-and-start-methods): > Changed in version 3.4: spawn added on all unix platforms, and forkserver added for some unix platforms. Child processes no longer inherit all of the parents inheritable handles on Windows. – Mike C Nov 17 '20 at 00:48