how to call stdout.write from subprocess.Popen?

Question

I'm trying to log a command executed with subprocess.Popen() to a UDP socket via the stdout. I've seen attempts on the net but none of real success at least in my case.

Some people tried to subclass a socket and pass it to the "stdout" argument of Popen in order to override the default .write method to send over a socket, see, e.g. http://www.dreamincode.net/forums/topic/209734-using-a-socket-as-stdinstdout/

I also tried to subclass the "file" class and override the write method with my implementation but Popen does not seem to call any write method at all.

Indeed this idea of subclassing doesn't work for the simple reason that Popen(args,stdout=mysocket) does not call any mysocket.write (or whatever .write) method. To test this I called Popen in debug mode and tried to step from its init to the end without finding any .write or something resembling it. I'm working with python 2.7

Of course I know I can send stdout to PIPE and then read it, but this would require in my case creating (more than one) working thread to read PIPE and send, and if possible I'd like to avoid this solution if a simpler one is available with lower effort.

my file subclass

class Foo(file):
    dbgIP = ""
    dbgPORT = 0
    sck = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)

    def __init__(self, dbgIP, dbgPORT):
        file.__init__(self,"empty",'w')
        self.sck.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 2)
        self.dbgIP = dbgIP
        self.dbgPORT = dbgPORT

    def write(self, text):
        return self.sck.sendto(text, (self.dbgIP, self.dbgPORT))

An example call

foofile = Foo(MCAST_GROUP,MCAST_PORT)
subprocess.Popen("jackd -R -P62 -dalsa -dhw:0 -p256 -n2 -i2 -o2 -s -S -r22050",shell=True, stdout=foofile, stderr=subprocess.STDOUT)

Calling foofile.write("asd") works fine

Answer 1

Popen() works at the file descriptors (handlers) level. It means that subclassing and overriding .write() method won't work. On some systems a real socket (valid .fileno()) can be passed to stdout as is:

from subprocess import call, STDOUT

rc = call("./generate-output", stdout=sck, stderr=STDOUT)

On other systems, you could do it by hand:

import shutil
from subprocess import Popen, PIPE, STDOUT

p = Popen("./generate-output", stdout=PIPE, stderr=STDOUT, bufsize=-1)
shutil.copyfileobj(p.stdout, foofile)
p.stdout.close()
rc = p.wait()

In more general case, you could use something similar to teed_call().

---

1) for usage of sockets as stdout, how to determine which systems will work? It depends on the Unix flavor/Win, or the python version?

I've tried it on Linux. I expect it to work on POSIX systems.

2) And if I want to subclass the sockets, what methods should I modify, e.g. to add a string to the string coming from stdout?

You should not subclass socket in this case. .fileno() is the only method that is called by subprocess. "works at the file descriptors level" means that once you give the file descriptor and the redirection occurs, the rest is the kernel business.

3) I assume the code with copyfileobj will initiate copying from stdout to foofile until the process is alive. However will closing stdout not stop that communication between the process and foofile? –

copyfileobj() copies until EOF in p.stdout. It is pure Python:

def copyfileobj(fsrc, fdst, length=16*1024):
    """copy data from file-like object fsrc to file-like object fdst"""
    while 1:
        buf = fsrc.read(length)
        if not buf:
            break
        fdst.write(buf)

p.stdout.close() is called to avoid leaking file descriptors (to cleanup).

There is a buffer so the copying may still occur after the subprocess terminates. And of cause the subprocess may close its stdout while it is still alive.