How do different processes share file descriptors?

Question

When forking a process, consider the following scenario:

1) We open two pipes for IPC bidirection communication
2) Suppose these have (3,4) and (5,6) as file descriptors.
3) We fork the process somewhere in the middle.
4) We exec the child process

Now, the thing that happens is that these two processes are completely independent of each other and the-then child process is now having it's own address space and is a completely new process.

Now, my question is, how do pipes(/file descriptor) live in an Execed processes? Because, pipes opened like this are used for the execed and the parent process to communicate.

The only way I see this could happen is when the file descriptors are global to the machine, which I think is impossible to happen, as that would be conflicting.

And in the IDE for this code:

import os

from multiprocessing import Process, Pipe


def sender(pipe):
    """
    send object to parent on anonymous pipe
    """

    pipe.send(['spam']+[42, 'eggs'])
    pipe.close()


def talker(pipe):
    """
    send and receive objects on a pipe
    """
    pipe.send(dict(name = 'Bob', spam = 42))
    reply = pipe.recv()
    print('talker got: ', reply)


if __name__ == '__main__':
    (parentEnd, childEnd) = Pipe()
    Process(target = sender, args = (childEnd,)).start()
    print("parent got: ", parentEnd.recv())
    parentEnd.close()

    (parentEnd, childEnd) = Pipe()
    child = Process(target = talker, args = (childEnd,))

    ##############################from here
    child.start()
    print('From talker Parent got:', parentEnd.recv())
    parentEnd.send({x * 2 for x in 'spam'})
    child.join()
    ############################## to here
    print('parent exit')

There are two processes run, but only the output from one process can be seen in the idle, not two processes. However, in the terminal, it's like the stdout is also shared.

Check this answer: https://stackoverflow.com/a/17741176/782570 — giusti, Sep 21 '17 at 11:48

myaut · Answer 1 · 2017-09-21T15:16:16.263

The actual job of copying process file descriptor table which is regulated by more generic clone() syscall flag CLONE_FILES (which is actually is not set by the fork()):

CLONE_FILES (since Linux 2.0)
...

If CLONE_FILES is not set, the child  process  inherits  a  copy  of  all  file
descriptors  opened in the calling process at the time of clone().  (The dupli-
cated file descriptors in the child refer to the same  open  file  descriptions
(see  open(2))  as  the corresponding file descriptors in the calling process.)
Subsequent operations that open or  close  file  descriptors,  or  change  file
descriptor  flags, performed by either the calling process or the child process
do not affect the other process.

execve() doesn't touch file descriptors except when file is opened or marked with O_CLOEXEC or FD_CLOEXEC flags, in which case those descriptors will be closed:

*  By default, file descriptors remain open across an execve().  File
descriptors that are marked close-on-exec are closed; see the
description of FD_CLOEXEC in fcntl(2).

I didn't understand much of the below highlighted technical stuff, but, I could gather that fork does not set this CLONE_FILES flag and the file descriptor are copied, but what about the exec-ing part. How would exec also copy the file descriptors to it? — mathmaniage, Sep 21 '17 at 13:52

How do different processes share file descriptors?

1 Answers1