Popen process not quitting / releasing when finished in Python

Question

I have a long running process which runs a command line application on PC. This command line application processes files and writes a log file into the directory it is reading from. This code is the portion which runs the application, i can run up to 4 processes at a time with threading:

command = "C:\\progra~1\\itms\\iTMSTransporter -m verify -f D:\\TEMP -u username -p password -o D:\\TEMP\\LOGFILE.txt -s provider -v eXtreme"
if slotNumber == 1:
    self.process1 = Popen(command, shell=True, stdin=PIPE)
elif slotNumber == 2:
    self.process2 = Popen(command, shell=True, stdin=PIPE)
elif slotNumber == 3:
    self.process3 = Popen(command, shell=True, stdin=PIPE)
elif slotNumber == 4:
    self.process4 = Popen(command, shell=True, stdin=PIPE)

After it has completed it runs this section:

if slotNumber == 1:
    wx.CallAfter(self.process1.kill)
elif slotNumber == 2:
    wx.CallAfter(self.process2.kill)
elif slotNumber == 3:
    wx.CallAfter(self.process3.kill)
elif slotNumber == 4:
    wx.CallAfter(self.process4.kill)

The only problem is that it is not releasing the directory and log file, If I try to move or rename the directory it says something is still using it. Looking at Windows Task Manager (screen grab below) it clearly shows lot of cmd.exe's and conhost.exe's. If I manually 'end process' for all these in Task Manager then the directory and log file is now relased and I can delete, move, rename, etc. If I run this application manually via command line from a command window when it completes the directory and log file is released straight away. Anyone have any idea why they are not being released in my Python code?

Task Manager Screen Grab

1) What is `wx.CallAfter`? 2) you say “After it has completed it runs this section: ...”. What is “it” in this sentence? 3) Why would you need to kill a process if it finished already? What makes you believe the process finished it’s job and needs manual killing? — Robert Siemer, May 21 '14 at 11:31
using shell=False gives me OSError: [Errno 2] No such file or directory error. — speedyrazor, May 21 '14 at 11:54
Use `os.killpg` as described in [How to terminate a python subprocess launched with shell=True][1] [1]: http://stackoverflow.com/questions/4789837/how-to-terminate-a-python-subprocess-launched-with-shell-true — Gabriel M, May 21 '14 at 12:06

score 0 · Answer 1 · answered May 26 '14 at 21:46

Consider using multiprocessing, and using myproc.join() to explicitly wait for each process to finish. Example:

from multiprocessing import Process

def f(name):
    print 'hello', name

if __name__ == '__main__':
    p = Process(target=f, args=('bob',))
    p.start()
    p.join()   # <=== wait for proc to finish

speedyrazor · Accepted Answer · 2014-05-27T18:47:39.423

I have figured it out, see below:

command = "C:\\progra~1\\itms\\iTMSTransporter.cmd -m verify -f D:\\TEMP -u username -p password -o D:\\TEMP\\LOGFILE.txt -s provider -v eXtreme"
startupinfo = subprocess.STARTUPINFO()
startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
if slotNumber == 1:
    self.process1 = Popen(command, startupinfo=startupinfo, shell=False)
elif slotNumber == 2:
    self.process2 = Popen(command, startupinfo=startupinfo, shell=False)
elif slotNumber == 3:
    self.process3 = Popen(command, startupinfo=startupinfo, shell=False)
elif slotNumber == 4:
    self.process4 = Popen(command, startupinfo=startupinfo, shell=False)

then to quit all the process's:

if slotNumber == 1:
    Popen("TASKKILL /F /PID {pid} /T".format(pid=self.process1.pid), startupinfo=startupinfo)
elif slotNumber == 2:
    Popen("TASKKILL /F /PID {pid} /T".format(pid=self.process2.pid), startupinfo=startupinfo)
elif slotNumber == 3:
    Popen("TASKKILL /F /PID {pid} /T".format(pid=self.process3.pid), startupinfo=startupinfo)
elif slotNumber == 4:
    Popen("TASKKILL /F /PID {pid} /T".format(pid=self.process4.pid), startupinfo=startupinfo)

This now correctly quits all the process's associated with each job.

Popen process not quitting / releasing when finished in Python

2 Answers2