4

So the title is a bit long but it is the only thing I cannot find online, with a little bit searching. How do I access the pass_fds argument from subprocess? 

# parent.py
import subprocess

subprocess.Popen(['run', 'some', 'program'], pass_fds=(afd, bfd))

# child.py
import subprocess

# need to access pass_fds argument? but how?
martineau
  • 119,623
  • 25
  • 170
  • 301
Yakup Türkan
  • 576
  • 2
  • 6
  • 21
  • so i guess only way to, without using some other ipc technique involved, is sending file descriptor to stdin of child process. And then the child would read from stdin and act according to received fd – Yakup Türkan Jan 02 '18 at 20:22
  • 1
    ...you *could* use whatever OS-specific interfaces (ie. procfs) to introspect what open FDs are available from the child and try to infer the interesting ones from metadata (ie. if you expect two TCP sockets, scanning for those), but that seems very error-prone, as opposed to having an explicit interface. – Charles Duffy Jan 02 '18 at 21:13
  • @charles sure normally i would use something like shared memory, but that did not seem python-ish and thought that there might be some info I missed. But after digging into cpython source code of `subprocess.Popen` I noticed they tried to be as general as possible, therefore they did not include a mechanism, if the said command is still calling python and do some (magic) trick to handle & store into predefined variables, etc. – Yakup Türkan Jan 03 '18 at 00:33
  • `subprocess` is intended to be used for invoking non-Python code. If you were creating subprocesses for the explicit purpose of running Python code in them, you'd be expected to use `multiprocessing`. – Charles Duffy Jan 03 '18 at 03:55

2 Answers2

5

You need to explicitly inform the child of the fds passed in some way. The most common/simple mechanisms would be:

  1. Via an environment variable set for the child
  2. Via an argument passed to the child
  3. (Less common, but possible) Written to the child's stdin

All of these require the child's cooperation of course; it needs to define an interface to inform it of the fds passed.

openssl's command line tool supports all these mechanisms for a similar purpose (communicating a passphrase to the child without putting it on the command line). You pass -pass and a second argument that defines where to look for the password. If the second argument is stdin, it reads from stdin, if it's -pass fd:# (where # is the fd number) it reads from an arbitrary file descriptor provided, -pass env:var (where var is the name of an environment variable) reads from the environment, etc.

ShadowRanger
  • 143,180
  • 12
  • 188
  • 271
  • i guess the easiest way, and probably safest way to, pass it via argument. as I already do some pre-processing on arguments anyway. thanks for the help – Yakup Türkan Jan 02 '18 at 20:45
-1

Well maybe it's not the best technical answer but I wanted to dig about file descriptors. Following your question I made this two scripts. Os creates pipe and parent sends descriptors to child by pass_fds tuple. More info in python docs , os.read() description. After that parent's fdw is pushed as bytes to child proc and then used to send back some answer.

Hope it helps.

edit: Found this post in google forums.

##### child.py
import subprocess, os

fdr_data = os.read(3,20) # 3 is inherited by pass_fds    
fdw = int(fdr_data)      # 4

print("CHILD fdw = ", fdw , "\n")

os.write(fdw, bytes("some answer".encode())) 
exit()


##### parent.py
import subprocess, os, time

fdr, fdw = os.pipe() # new file descriptor read , fd write
print("PARENT", "fdr = ", fdr , " fdw = " , fdw)

subprocess.Popen(['python3','child.py'], pass_fds=(fdr, fdw))    
os.write(fdw, bytes("{}".format(fdw).encode())) # pipe file descriptor write (out 4)

time.sleep(1)   # so subproc can execute

read_pipe = os.read(fdr, 20) # pipe file descriptor read (in 3)    
print("PARENT" , read_pipe.decode())
Tomasz Wiśniewski
  • 129
  • 5