Using subprocess to run Python script on Windows

Question

Is there a simple way to run a Python script on Windows/Linux/OS X?

On the latter two, subprocess.Popen("/the/script.py") works, but on Windows I get the following error:

Traceback (most recent call last):
  File "test_functional.py", line 91, in test_functional
    log = tvnamerifiy(tmp)
  File "test_functional.py", line 49, in tvnamerifiy
    stdout = PIPE
  File "C:\Python26\lib\subprocess.py", line 595, in __init__
    errread, errwrite)
  File "C:\Python26\lib\subprocess.py", line 804, in _execute_child
    startupinfo)
WindowsError: [Error 193] %1 is not a valid Win32 application

---

monkut's comment: The use case isn't clear. Why use subprocess to run a python script? Is there something preventing you from importing the script and calling the necessary function?

I was writing a quick script to test the overall functionality of a Python-command-line tool (to test it on various platforms). Basically it had to create a bunch of files in a temp folder, run the script on this and check the files were renamed correctly.

I could have imported the script and called the function, but since it relies on sys.argv and uses sys.exit(), I would have needed to do something like..

import sys
import tvnamer
sys.argv.append("-b", "/the/folder")
try:
    tvnamer.main()
except BaseException, errormsg:
    print type(errormsg)

Also, I wanted to capture the stdout and stderr for debugging incase something went wrong.

Of course a better way would be to write the script in more unit-testable way, but the script is basically "done" and I'm doing a final batch of testing before doing a "1.0" release (after which I'm going to do a rewrite/restructure, which will be far tidier and more testable)

Basically, it was much easier to simply run the script as a process, after finding the sys.executable variable. I would have written it as a shell-script, but that wouldn't have been cross-platform. The final script can be found here

Answer 1

Just found sys.executable - the full path to the current Python executable, which can be used to run the script (instead of relying on the shbang, which obviously doesn't work on Windows)

import sys
import subprocess

theproc = subprocess.Popen([sys.executable, "myscript.py"])
theproc.communicate()

Answer 2

How about this:

import sys
import subprocess

theproc = subprocess.Popen("myscript.py", shell = True)
theproc.communicate()                   # ^^^^^^^^^^^^

This tells subprocess to use the OS shell to open your script, and works on anything that you can just run in cmd.exe.

Additionally, this will search the PATH for "myscript.py" - which could be desirable.

Answer 3

Yes subprocess.Popen(cmd, ..., shell=True) works like a charm. On Windows the .py file extension is recognized, so Python is invoked to process it (on *NIX just the usual shebang). The path environment controls whether things are seen. So the first arg to Popen is just the name of the script.

subprocess.Popen(['myscript.py', 'arg1', ...], ..., shell=True)

Answer 4

It looks like windows tries to run the script using its own EXE framework rather than call it like

python /the/script.py

Try,

subprocess.Popen(["python", "/the/script.py"])

Edit: "python" would need to be on your path.

Answer 5

For example, to execute following with command prompt or BATCH file we can use this:

C:\Python27\python.exe "C:\Program files(x86)\dev_appserver.py" --host 0.0.0.0 --post 8080 "C:\blabla\"

Same thing to do with Python, we can do this:

subprocess.Popen(['C:/Python27/python.exe', 'C:\\Program files(x86)\\dev_appserver.py', '--host', '0.0.0.0', '--port', '8080', 'C:\\blabla'], shell=True)

or

subprocess.Popen(['C:/Python27/python.exe', 'C:/Program files(x86)/dev_appserver.py', '--host', '0.0.0.0', '--port', '8080', 'C:/blabla'], shell=True)

Answer 6

You are using a pathname separator which is platform dependent. Windows uses "\" and Unix uses "/".

Answer 7

When you are running a python script on windows in subprocess you should use python in front of the script name. Try:

process = subprocess.Popen("python /the/script.py")

Answer 8

Supplemental info: It is worth noting that the documentation states that you need to use shell=True if you are using a dos shell command like dir

without it you get something like this.

>>> import subprocess
>>> subprocess.run(['dir'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\foo\AppData\Local\Programs\Python\Python38\lib\subprocess.py", line 489, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\Users\foo\AppData\Local\Programs\Python\Python38\lib\subprocess.py", line 854, in __ini
t__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Users\foo\AppData\Local\Programs\Python\Python38\lib\subprocess.py", line 1307, in _exe
cute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified
>>> subprocess.run(['dir'], shell=True)
 Volume in drive J is garbage
 Volume Serial Number is 5EE7-B084

Also you can use path like objects for the args which is recent addition.

from pathlib import Path 
subprocess.run(Path('c:/proj/myfile.bat'))

Also worth noting there is a whole set of windows specific controls that allow you to control how a process is spawned which concurrent operations can use.

So controlling subprocesses on windows is not as simple as posix style.