Start another process in background and capture output in Python

Question

In python I want to start another python script as background process and after few sec I need to kill that spawned process and get the stdout into a variable.

I tried with subprocess.Popen, I am able to spawn as background process and kill after few secs. But atlast while redirecting stdout to a variable it gets blocked.

Can someone suggest me to fix it ? Or is there any other module available to do this other than subprocess.Popen ?

g_flag = 0
class StartChecking(Thread):
    def __init__(self):
        Thread.__init__(self)
    def run(self):
        global g_flag
        print 'Starting thread'
        proc = subprocess.Popen(["./cmd.py"], stdout=subprocess.PIPE, shell=True)
        pid = proc.pid
        print 'Sniff started ' + str(pid)
        if (pid != 0) and (pid != 1):
            while g_flag == 0:
                sleep(0.1)
            os.kill(pid, signal.SIGTERM)
            print 'Killed ' + str(pid)
        (out, err) = proc.communicate() # Currently its blocking here
        print out


th = StartChecking()
th.start()
#Do something else
sleep(5)
g_flag = 1
th.join()
print 'thread joined'

Output is

Starting thread
Sniff started 24294
Killed 24294

Note : Using Python 2.7.12 in ubuntu 16.04.

Don't mix threads and fork http://www.linuxprogrammingblog.com/threads-and-fork-think-twice-before-using-them — Vorsprung, Mar 26 '18 at 13:50
Ya correct. Initially I was using without thread, later for fixing that block issue I tried something with thread. — rashok, Mar 26 '18 at 14:13

score 1 · Accepted Answer · answered Mar 26 '18 at 14:13

After removing shell=True it started working. And restructured the code without thread also. If I add shell=True then it again starts blocking in communicate call.

proc = subprocess.Popen(["./cmd.py"], stdout=subprocess.PIPE)
#Do something else
sleep(2)
os.system('kill -15 ' + str(proc.pid))
print 'Killed ' + str(proc.pid)
print 'cmd out: ' + proc.communicate()[0]
print 'finished'

dheeraj tripathi · Answer 2 · 2018-03-26T18:31:39.293

Here is a good explaination of shell=True

Setting the shell argument to a true value causes subprocess to spawn an intermediate shell process, and tell it to run the command

Always use=shell=False, your code however works on my linux environment as below, for simplicity I used a shell file as command to run] :

import os,signal
import time
import subprocess
from threading import Thread

g_flag = 0
class StartChecking(Thread):
    def __init__(self):
        Thread.__init__(self)
    def run(self):
        global g_flag
        print 'Starting thread'
        proc = subprocess.Popen(["./a.sh"], stdout=subprocess.PIPE, shell=True)
        pid = proc.pid
        print 'Sniff started ' + str(pid)
        if (pid != 0) and (pid != 1):
            while g_flag == 0:
                time.sleep(0.1)
            os.kill(pid, signal.SIGTERM)
            print 'Killed ' + str(pid)
        (out, err) = proc.communicate() # Currently its blocking here
        print out


th = StartChecking()
th.start()
#Do something else
time.sleep(5)
g_flag = 1
th.join()
print 'thread joined'

As you pointed out shell=True is actually the issue, always prefer shell=False, your code however works on my Linux environment — dheeraj tripathi, Mar 26 '18 at 18:25

Start another process in background and capture output in Python

2 Answers2