python daemon thread exits but process still run in the background

Question

I am using python 2.7 and Python thread doesn't kill its process after the main program exits. (checking this with the ps -ax command on ubuntu machine)

I have the below thread class,

import os
import threading

class captureLogs(threading.Thread):

'''
initialize the constructor
'''
def __init__(self, deviceIp, fileTag):
    threading.Thread.__init__(self)
    super(captureLogs, self).__init__()
    self._stop = threading.Event()
    self.deviceIp = deviceIp
    self.fileTag = fileTag

def stop(self):
    self._stop.set()

def stopped(self):
    return self._stop.isSet()
'''
define the run method
'''
def run(self):
    '''
    Make the thread capture logs
    '''
    cmdTorun = "adb logcat > " + self.deviceIp +'_'+self.fileTag+'.log'
    os.system(cmdTorun)

And I am creating a thread in another file sample.py,

import logCapture
import os
import time

c = logCapture.captureLogs('100.21.143.168','somefile')
c.setDaemon(True)
c.start()

print "Started the log capture. now sleeping.  is this a dameon?", c.isDaemon()
time.sleep(5)
print "Sleep tiime is over"
c.stop()

print "Calling stop was successful:", c.stopped()
print "Thread is now completed and main program exiting"

I get the below output from the command line:

Started the log capture. now sleeping.  is this a dameon? True
Sleep tiime is over
Calling stop was successful: True
Thread is now completed and main program exiting

And the sample.py exits. But when I use below command on a terminal,

ps -ax | grep "adb"

I still see the process running. (I am killing them manually now using the kill -9 17681 17682)

Not sure what I am missing here.

My question is, 1) why is the process still alive when I already killed it in my program?

2) Will it create any problem if I don't bother about it?

3) is there any other better way to capture logs using a thread and monitor the logs?

EDIT: As suggested by @bug Killer, I added the below method in my thread class,

def getProcessID(self):
        return os.getpid()

and used os.kill(c.getProcessID(), SIGTERM) in my sample.py . The program doesn't exit at all.

Usually if you daemonize a process it continues to run in the background even after the parrent process exits — Bug Killer, Oct 15 '15 at 23:58
Try adding signal(SIGTERM, lambda signum, stack_frame: exit(1)) to your daemon code. If that doesn't work, try SIGKILL — Bug Killer, Oct 16 '15 at 00:11

score 1 · Accepted Answer · answered Oct 17 '15 at 04:01

It is likely because you are using os.system in your thread. The spawned process from os.system will stay alive even after the thread is killed. Actually, it will stay alive forever unless you explicitly terminate it in your code or by hand (which it sounds like you are doing ultimately) or the spawned process exits on its own. You can do this instead:

import atexit
import subprocess

deviceIp = '100.21.143.168'
fileTag = 'somefile'

# this is spawned in the background, so no threading code is needed
cmdTorun = "adb logcat > " + deviceIp +'_'+fileTag+'.log'
proc = subprocess.Popen(cmdTorun, shell=True)

# or register proc.kill if you feel like living on the edge
atexit.register(proc.terminate)

# Here is where all the other awesome code goes

Since all you are doing is spawning a process, creating a thread to do it is overkill and only complicates your program logic. Just spawn the process in the background as shown above and then let atexit terminate it when your program exits. And/or call proc.terminate explicitly; it should be fine to call repeatedly (much like close on a file object) so having atexit call it again later shouldn't hurt anything.

thanks a log for the suggestion.. this is partially correct.. it will kill one of the process but the other one still remains. — Harish Talanki, Oct 19 '15 at 17:28
Interesting. Is the plain `adb logcat` a daemon that gets spawned when you run `"adb logcat > " + self.deviceIp +'_'+self.fileTag+'.log'`? You only explicitly spawn one process so I am left to believe that that process spawns yet another process (or starts a daemon, however, the two PIDs are sequential, so it looks more like a plain old spawn). — eestrada, Oct 19 '15 at 20:17
Yes you are right :) the command that I am running spawns another process which is plain 'adb logcat'. So I think this is the final solution. If I don't get any other replies, I will select your answer as the solution. :) — Harish Talanki, Oct 19 '15 at 22:25

python daemon thread exits but process still run in the background

1 Answers1

Linked