0

I want to read on-going output of started process from code (running in background) with subprocess.Popen() and subprocess.communicate()

Starting the process:

import subprocess
process_params = ['/usr/bin/tcpdump', '-n', 'dst port 80']

proc = subprocess.Popen(
    process_params,
    stdout=subprocess.PIPE, stderr=subprocess.PIPE
)

(stdout, stderr) = proc.communicate()

The tcpdump process is running in background but proc.communicate() waits till end of file and only when process is killed it produces some output.

>>> (stdout, stderr) = proc.communicate()

I would like to achieve something like receiving data from process' stdout at the moment when output is produced by the process. I think I need some thread that looks if some output is generated from process and then for example append it to log file. I don't know how to get down to it, so any ideas and suggestions will be much appreciated.

deveraja
  • 1
  • Have a look at my answer https://stackoverflow.com/a/49209440/7738328. There the `stdout` stream is caught in a thread and modified before written to a log. That is similar to your use case, I assume? – JohanL Mar 17 '18 at 13:35

1 Answers1

0

tcpdump is probably buffering its output. It will only write output when it has a buffer full of data (typically 8KB) to write. This is common behavior for programs which are not writing to a TTY.

Tcpdump has a command-line option to line-buffer its output. This causes it to write its output every time it has a full line of text. Try this:

process_params = ['/usr/bin/tcpdump', '-l', '-n', 'dst port 80']
                                       ^^--Line buffer output

Alternately, see if you have programs named stdbuf or unbuffer installed on your system. They can be used to adjust the buffering behavior of another process. Or see these questions:

Kenster
  • 23,465
  • 21
  • 80
  • 106
  • While this is true, making tcpdump unbuffered will not fix the issue, since p.communicate won't return until the subprocess finishes. Thus, there is still the need for a threaded solution, for the `stdout` output. – JohanL Mar 17 '18 at 09:46
  • Thanks Kenster for your idea of -l parameter, with combination of JohanL answer the tcpdump output is correctly saving to file :) – deveraja Mar 18 '18 at 08:42