Accessing file descriptors created by a child process with subprocess.Popen in Python 2.5

Question

is it possible for Python 2.5 to access the file descriptors opened by subprocess.Popen() child processes? Not specifically the input and output, but other pipes that it creates? If so, how would I do this? Appreciate the help.

Answer 1

There is no cross-platform way to access another process's file descriptors.

On almost any POSIX system, you can use sendmsg to pass open files over a Unix socket or pipe (usually requiring SCM_RIGHTS), and Windows has different but similar functionality. So, if you control the child process, you can do things that way. This allows you to have the same actual file object and/or a clone of it, although you will still have a different descriptor (small number) or handle for that object.

Some platforms have a way to access another process's file handles explicitly by using their descriptor or handle. On linux, for example, '/proc/{}/fd/{}'.format(childpid, childfd) will be a symlink to the open file (even if it's not in a name-accessible part of the filesystem), although of course in that case you end up with a different file object for the same file. Windows has NT-level APIs for enumerating all open handles of a child process; you usually won't have access to them unless the child gives it to you explicitly, but you can get the pathname and open it for yourself. (Although any close-on-exec files will get broken, of course, so be careful with that… and if you want to use stdin/out/err pipes it will get a lot more complicated.)

Finally, some platforms have ways to do something between forking a new process and just starting a new thread. For example, with linux, you can clone with all the flags the same as in fork, except for CLONE_FILES set to true, and you will then have a separate process that nevertheless shares the file descriptor table. Then you can actually use the same file descriptor numbers to refer to the same file objects. This obviously isn't wrapped up by subprocess, but it wouldn't be too hard to fork the subprocess source (or, better, if you're using 3.1 or earlier, the subprocess32 backport) and make your own version that wraps this up.

If you want a cross-platform solution, you can build one yourself. For example, in most cases, you can just pass an absolute pathname over a pipe, and have the parent open the same file as the child. Unless you really need to share the position within the file or similar, or are using file descriptors to files you don't have access to as passable capabilities (and in either case you're probably not writing cross-platform code), this will usually work just as well, and it's a lot simpler.