15

I'm writing a programme that needs to run on both Linux and Windows and use executables (with parameters) that exist in the path. (Assumed)

Currently I'm having trouble running executables in windows using Subprocess.Call and Subprocess.Popen.

For a code like this, in windows 8

def makeBlastDB(inFile, inputType, dbType, title, outDir):
    strProg = 'makeblastdb'
    strInput = '-in ' + inFile
    strInputType = '-input_type ' + inputType
    strDBType = '-dbtype ' + dbType
    strTitle = '-title ' + title
    strOut = '-out ' + os.path.join(os.sep, outDir, title)
    cmd = [strProg, strInput, strInputType, strDBType, strTitle, strOut]
    result = Popen(cmd, shell=True)

I get the error message in console

'makeblastdb' is not recognized as an internal or external command,
operable program or batch file.

Even though I can run the same command using cmd.exe I get the same response with shell=False.

Any ideas on how I can run the command assuming that the executable is in PATH environment variable? Thanks

Chris_Rands
  • 38,994
  • 14
  • 83
  • 119
Jacob Wang
  • 4,411
  • 5
  • 29
  • 43
  • Just make a simple script to add the executable to your path or you can provide the full path to the executable. –  Feb 04 '13 at 01:57
  • I don't really want to go that route because the executable is third party, which means it could be installed anywhere. – Jacob Wang Feb 05 '13 at 01:05
  • Does it work in the shell? Have you compared `echo %PATH%` and `os.environ['PATH']`? – jfs May 10 '16 at 18:44
  • @J.F.Sebastian the question is over 3 years old and OP already solved their problem somehow (with a dubious answer), I'm not sure you will get more information from here... ;) It was probably either a different cwd or some bad windows weirdness, because the accepted answer doesn't make much sense. – wim May 10 '16 at 18:52
  • @wim: if OP won't update the question then it should be closed as non-reproducible (because obviously `subprocess.Popen` can run (at least some) executables that are in PATH on Linux and Windows). – jfs May 10 '16 at 19:12

4 Answers4

9

You can control the environment variables available in the spawned subprocess by passing a mapping with the env keyword argument. E.g.

proc = subprocess.Popen(args, env={'PATH': '/some/path'})

Or to inherit PATH from the system environment variable, without necessarily chucking in everything else from the system environment:

proc = subprocess.Popen(args, env={'PATH': os.getenv('PATH')})

It might be easier/simpler just to use an absolute path, though.

wim
  • 338,267
  • 99
  • 616
  • 750
  • Thanks. with regards to absolute path, since the program will be running presumaably on machines that already has the .exe installed, so I didn't want to go through the trouble to manually find the .exe. – Jacob Wang Feb 04 '13 at 04:32
  • Didn't work for me. I will come back to this later and answer it myself if I find the solution. – Jacob Wang Feb 07 '13 at 12:50
  • There's an error in the second example. `os.environ()['PATH']` should be replaced with `os.environ['PATH']`. – Wyatt Baldwin May 09 '16 at 19:30
  • @WyattBaldwin Thanks, fixed. Feel free to directly edit answers yourself, in future. – wim May 09 '16 at 20:32
  • `env={'PATH': os.getenv('PATH')}` seems useless (PATH is inherited anyway) or even harmful if the child may require other envvars. If OP knows the path; OP could pass the full path (including the file extension) instead (in `args[0]`). – jfs May 10 '16 at 18:42
  • Hmm, yeah, but reading the answer on it's own is lacking some context. This was more about restricting the environment to just the ones needed - trying to replace the `shell=True` that the OP had in the question. I'll edit it to make that clearer. – wim May 10 '16 at 18:47
  • I don't see any context where the answer is useful. Is child's `PATH` even used to find the executable? (I would expect only `os.environ['PATH']` in the script that calls `subprocess.Popen` is relevant (plus [Windows' idiosyncratic algorithm to find the executable](http://stackoverflow.com/a/25167402/4279)). – jfs May 10 '16 at 19:06
  • Yes, the child's PATH is used to find the executable. Send `env={}` and see for yourself. – wim May 10 '16 at 19:46
  • Doesn't work if system path is set by sys.path.append(...) – tribbloid Jan 05 '17 at 17:23
  • I was having the same problem until I read: https://stackoverflow.com/a/17046730, so keep that in mind. – joejoejoejoe4 Jan 23 '21 at 10:47
6

Ok here is how I got it to work.

env = os.environ
proc = subprocess.Popen(args, env=env)
Jacob Wang
  • 4,411
  • 5
  • 29
  • 43
  • 3
    If I'm not mistaken, this has no net effect, since "the default behavior of [Popen is] inheriting the current process’ environment". – Wyatt Baldwin May 09 '16 at 19:34
  • @WyattBaldwin: you are not mistaken. It is easy to check, run: `Popen([sys.executable, '-c', 'import os; print(os.environ)'])`, to see that the environment is inherited. You might want to pass `env` if you want to modify the child environment: `env=dict(os.environ, ENVVAR='visible in only in child')` – jfs May 10 '16 at 18:38
5

I struggled with this myself until I found this python bug report.

"If you add a directory into PATH on Windows so that the directory is in quotes, subprocess does not find executables in it." Since the quotes aren't required by windows removing them fixes my problem (in 2.7).

joshua
  • 2,509
  • 17
  • 19
1

A colleague of mine has reproduced this issue with Python 3.6.5 on Windows 10 64-bits.

The installed version of Python was however 32-bits.

Re-installing 64-bits version of Python fixed this issue.

Ryuu
  • 473
  • 2
  • 6
  • 11