0

I am trying to periodically read the most recent output of a subprocess, to then process further elsewhere. I am not that experienced in subprocess, maybe you have some best practices for me? The steps are:

  • Start the process: This process should run in the background, so I will not see its whole output on the python terminal (I am afraid, that something is going to crash due to holding a lot of data in any variables of the python script). The process will produce data every second (a line with about 30 characters).
  • Read latest data: In an infinite while-loop of the python script I like to get the latest line of the subprocess every 3 seconds.
import subprocess
import os
import time

os.chdir('/home/pi/Tutorials/RIOT/examples/gnrc_networking')
cmd = ["sudo", "BOARD=pba-d-01-kw2x", "make", "all", "flash", "term"]
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
while True:
    line = p.stdout.readline().rstrip()
    print(line)
accdias
  • 5,160
  • 3
  • 19
  • 31
paranoia_do
  • 1
  • 2
  • 2
    Please show your efforts. I hope you have tried to implement it? It is easier to answer when there is something concrete to build on. – Jaideep Shekhar Jan 09 '20 at 15:55
  • 1
    Rough idea: use two threads and a [`deque`](https://docs.python.org/3.8/library/collections.html#collections.deque) with maximum size one. One thread continuously copies lines from the subprocess to the deque, so that the deque always contains exactly one line, the newest. The other thread reads from the `deque` as needed. You'll have to implement some sort of locking so that the two threads don't try to access the deque at the same time. – chepner Jan 09 '20 at 16:04
  • If the child process does not use buffering for its output, it should work without any major problem: the magic of pipes will suspend the write end if the pipe buffer is full, so that the read can process data when it wants. The real problem is that many processes use buffering when non writing to a terminal which will cause the receiver to wait until a full output buffer is ready (commonly several ko or Mo) – Serge Ballesta Jan 09 '20 at 16:06
  • Not sure, but check this issue. Perhaps it might help: https://stackoverflow.com/questions/13331940/using-python-popen-to-read-the-last-line – Kalma Jan 09 '20 at 16:35
  • An alternative to `line = p.stdout.readline().rstrip()` is the standard library module [`fileinput`](https://docs.python.org/3/library/fileinput.html). – accdias Jan 09 '20 at 23:04

0 Answers0