2

I have this code 

fd_log_out = open(path_log_out, 'w')
fd_log_err = open(path_log_err, 'w')
subprocess.check_call(command, cwd=path, stdout=fd_log_out, stderr=fd_log_err)

I would like to process the stderr during the check_call to trigger an event if something is seen. I have tried to create a subclass of TextIOWrapper and overwrite the write function but it was never called.

What is the called function by subprocess.check_call. The source code is too complex to find it.

Is there any other way to do this? With PIPE maybe ?

Thanks.

Benjamin
  • 3,350
  • 4
  • 24
  • 49
  • The whole point of a redirection is that it's changing where a FD writes to. If you pass an object that has a FD number (that is, where it has a `fileno()` method) as `stdout` or `stderr`, the output from the process goes *straight* to that FD -- the Python interpreter never sees it at all, and has no opportunity to process it. – Charles Duffy Jun 20 '17 at 17:46
  • The idea was to analyse lines of a wrtie call, check, call super.write. – Benjamin Jun 20 '17 at 17:48
  • As I said, it's bypassing the interpreter at an OS level; the data isn't exposed to Python at all. (Which is to say, yes, you'd need to use `subprocess.PIPE` to implement a shim). – Charles Duffy Jun 20 '17 at 17:49
  • Related: [How to replicate tee behavior in Python when using subprocess?](https://stackoverflow.com/q/2996887/674039) – wim Jun 20 '17 at 17:59

1 Answers1

2

When an object with a fileno() method is passed as an argument to stdout= or stderr= of subprocess.Popen(), then the relevant file descriptor is redirected with fdup2() or a similar syscall.

This means the data is sent directly from the subprocess to the file associated with the FD -- it isn't read by the Python interpreter at all, so the Python interpreter never calls write() on an associated object.

To work around this, you do indeed need to use subprocess.PIPE -- and then have an ongoing process shuffling data on both stdout and stderr (you can't do it one-after-the-other without causing deadlocks) through analysis and to their eventual destinations.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441