34

I'm trying to launch a completely independent process from python. I can't use something simple like os.startfile since I need to pass arguments. Currently I'm using subprocess.popen which gets me 90% of the way there.

args = ["some_exe.exe", "some_arg", "another_arg"]
subprocess.Popen(args, creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)

Using popen with detached creation flags & pipes for std* does start a new process that lives after the parent process dies. So thats all good. The problem is that the new 'child' process still holds a phantom handle to the parent. So if I try to uninstall the parent exe (my python script is bundled into an exe via pyinstaller) msiexec complains that the parent exe is still in use.

So the goal is to spawn a totally independent process to run "some_exe.exe" that doesn't have any handle back to the original process.

Note: This is for Windows XP and above. I'm developing on Win7.

djvg
  • 11,722
  • 5
  • 72
  • 103
greenhat
  • 1,061
  • 1
  • 12
  • 19
  • 2
    Have you looked at http://stackoverflow.com/questions/204017/how-do-i-execute-a-program-from-python-os-system-fails-due-to-spaces-in-path and http://stackoverflow.com/questions/1196074/starting-a-background-process-in-python ? I have no time to try it out for myself... – MartinStettner Nov 27 '12 at 20:16
  • I looked at that post but I ended up using popen because, as one of the comments says, it is the new preferred way (according to docs. I have no idea why). However you're right I should (and am about to) try spawnl too. – greenhat Nov 27 '12 at 20:38
  • Please report on the results of your experiments :) This might well be of interest to me in some future projects ... – MartinStettner Nov 27 '12 at 20:41
  • 1
    huh... i'm running into crashes using spawnl (maybe why it is recommended not to use?). Some ppl on the earlier thread you linked were getting crashes too. Just running "os.spawnl(os.P_NOWAIT, r"c:\windows\notepad.exe")" from the python interpreter killed python for me. P_DETACH also gets a crash as does P_WAIT. – greenhat Nov 27 '12 at 21:00
  • The issue may be with how your Python exe is packaged, e.g. it's maintaining a handle to one of the parent's DLLs -- something along those lines. You can examine the open handles with [ProcessExplorer](http://technet.microsoft.com/en-gb/sysinternals/bb896653.aspx). Just speculation, but it's possible that pyinstaller is bundling some DLL that gets included on the `PATH` of the child process and therefore gets loaded. – ig0774 Nov 27 '12 at 21:35

2 Answers2

28

I think I found the answer. By using Popen with close_fds = True I was able to start up a process that was independent and without handles to the parent.

For docs look here and search for close_fds.

Or, on Windows, if close_fds is true then no handles will be inherited by the child process. Note that on Windows, you cannot set close_fds to true and also redirect the standard handles by setting stdin, stdout or stderr.

Note this solution only works on Windows. I have no idea about any *nix system.

Felipe
  • 3,003
  • 2
  • 26
  • 44
greenhat
  • 1,061
  • 1
  • 12
  • 19
  • 2
    I've been trying to start a daemon process using a python script called from within a makefile, but make.exe wouldn't exit until this daemon exited itself, which was not what I wanted. What worked eventually was a combination of `close_fds=True` **and** `creationflags=DETACHED_PROCESS`. Conclusion: use them both, e.g.: `subprocess.Popen(, close_fds=True, creationflags=DETACHED_PROCESS)` – Zuzu Corneliu Nov 10 '15 at 21:16
  • In addition to @CorneliuZuzu's suggestion, I used `CREATE_NEW_PROCESS_GROUP = 0x00000200` with `DETACHED_PROCESS = 0x00000008`: `creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP, close_fds=True` – jtpereyda Aug 11 '17 at 17:07
  • Note that starting from Python 3.7, on Windows it's no longer necessary to set `close_fds=True` - that is now the default and works together with redirecting `stdin`/`stdout`/`stderr`. Details in bug tracker here: https://bugs.python.org/issue19764 – nirvana-msu Jan 06 '20 at 13:01
4

I found this here:

On windows (win xp), the parent process will not finish until the longtask.py has finished its work. It is not what you want in CGI-script. The problem is not specific to Python, in PHP community the problems are the same.

The solution is to pass DETACHED_PROCESS Process Creation Flag to the underlying CreateProcess function in win API. If you happen to have installed pywin32 you can import the flag from the win32process module, otherwise you should define it yourself:

DETACHED_PROCESS = 0x00000008

pid = subprocess.Popen([sys.executable, "longtask.py"], creationflags=DETACHED_PROCESS).pid

rstackhouse
  • 2,238
  • 24
  • 28
f p
  • 3,165
  • 1
  • 27
  • 35
  • Doesn't he indicate that he does this? – ig0774 Nov 27 '12 at 21:26
  • I actually currently am using the detached flag (check out the code example). That flag does work to create a detached process but it still give the child a handle to the parent. – greenhat Nov 27 '12 at 21:43