Python - Launch a Long Running Process from a Web App

Question

I have a python web application that needs to launch a long running process. The catch is I don't want it to wait around for the process to finish. Just launch and finish.

I'm running on windows XP, and the web app is running under IIS (if that matters).

So far I tried popen but that didn't seem to work. It waited until the child process finished.

instead of writing "that didn't seem to work", try writing what exactly happened. Did it raise an exception? Did it do something other than what you expected? (if so, what did it do?). This will usually make it easier for other people to help you. — Ofri Raviv, Jun 03 '10 at 22:04
Here are a few windows API commands that look promising, I'll try them out if we don't come up with anything else: http://timgolden.me.uk/pywin32-docs/win32api__ShellExecute_meth.html http://timgolden.me.uk/pywin32-docs/win32console__FreeConsole_meth.html http://timgolden.me.uk/pywin32-docs/win32process__CreateProcess_meth.html — Greg, Jun 03 '10 at 22:10

score 7 · Accepted Answer · edited Jul 28 '17 at 04:22

7

Ok, I finally figured this out! This seems to work:

from subprocess import Popen
from win32process import DETACHED_PROCESS

pid = Popen(["C:\python24\python.exe", "long_run.py"],creationflags=DETACHED_PROCESS,shell=True).pid
print pid
print 'done' 
#I can now close the console or anything I want and long_run.py continues!

Note: I added shell=True. Otherwise calling print in the child process gave me the error "IOError: [Errno 9] Bad file descriptor"

DETACHED_PROCESS is a Process Creation Flag that is passed to the underlying WINAPI CreateProcess function.

edited Jul 28 '17 at 04:22

rstackhouse

2,238
24
28

answered Jun 04 '10 at 12:41

Greg

45,306
89
231
297

1

Any idea on how to do this on Unix? – nbarraille Jul 15 '11 at 12:49
This flag does not seem needed on Unix – MarcH Jul 23 '12 at 13:07
1

I don't need shell=True for this to work. The bad file descriptor error could be unrelated. – MarcH Jul 23 '12 at 13:08

score 2 · Answer 2 · answered Jun 04 '10 at 09:24

2

Instead of directly starting processes from your webapp, you could write jobs into a message queue. A separate service reads from the message queue and runs the jobs. Have a look at Celery, a Distributed Task Queue written in Python.

answered Jun 04 '10 at 09:24

Daniel Hepper

28,981
10
72
75

score 1 · Answer 3 · answered Jun 04 '10 at 12:04

This almost works (from here):

from subprocess import Popen

pid = Popen(["C:\python24\python.exe", "long_run.py"]).pid
print pid
print 'done'

'done' will get printed right away. The problem is that the process above keeps running until long_run.py returns and if I close the process it kills long_run.py's process.

Surely there is some way to make a process completely independent of the parent process.

score 0 · Answer 4 · answered Jun 03 '10 at 22:00

0

subprocess.Popen does that.

answered Jun 03 '10 at 22:00

Ofri Raviv

24,375
3
55
55

1

Do you know how I would call it? When I tried it, it waited until the long running process finished. Maybe I was calling it wrong. – Greg Jun 03 '10 at 22:12
You must not wait for the application output or termination :) – Messa Jun 04 '10 at 12:06

Python - Launch a Long Running Process from a Web App

4 Answers4

Linked