Popen error: "[Errno 2] No such file or directory" when calling shell function

Question

I have some custom commands.

This works:

subprocess.Popen(['python'], stdout=subprocess.PIPE)

But if I have my own system commands like deactivate, I get that error

Traceback (most recent call last):
  File "runner2.py", line 21, in <module>
    main() 
  File "runner2.py", line 18, in main
    subprocess.Popen(['deactivate',], stdout=subprocess.PIPE)
  File "/usr/lib/python2.6/subprocess.py", line 633, in __init__
    errread, errwrite)
  File "/usr/lib/python2.6/subprocess.py", line 1139, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Let alone I need to execute this under my sandbox virtualenv.

Maybe `python` is not on the `PATH` environment variable when your script runs. Try setting the full path to python, i.e. `/usr/bin/python`. — El Barto, Mar 29 '12 at 23:44
Can you explain what you're trying to accomplish? I suspect that the child shell you're launching with `subprocess` hasn't "sourced" the virtualenv activation script, and it's not inherited from the parent Python process (assuming that's where you're running it from). — Brendan Wood, Mar 29 '12 at 23:45

score 145 · Answer 1 · edited Feb 08 '21 at 15:22

145

Try add an extra parameter shell=True to the Popen call.

edited Feb 08 '21 at 15:22

Neuron

5,141
5
38
59

answered Mar 30 '12 at 00:29

Anthony Kong

37,791
46
172
304

53

Worked for me, but why? – YMomb Mar 11 '15 at 17:44
8

@YMomb: `deactivate` is a shell function. To run it, you need a shell. Though it is pointless to try to run it in a new shell (the new shell probably won't have it defined until `venv/bin/activate` is called and the child normally can't modify its parent environment anyway if OP hopes to deactivate the current virtualenv set in the parent shell. It is the same reason why `subprocess.call('cd')` raises "No such file or directory" and it can be fixed using `shell=True` and it would be equally pointless.See [Why is cd not a program?](http://unix.stackexchange.com/q/38808/1321) – jfs May 20 '16 at 09:32
It didn't work for me. The same script is working fine when executed on shell, but gives error on subprocess. The error is : " python3: can't open file '$SDE/install/lib/python3.6/site-packages/p4testutils/bf_switchd_dev_status.py': [Errno 2] No such file or directory " What may be the issue? – Nagmat Sep 15 '22 at 22:12

score 60 · Answer 2 · edited Jul 19 '17 at 09:28

60

Just a note. shell=True was likely the correct solution to the o.p., since they did not make the following mistake, but you can also get the "No such file or directory" error if you do not split up your executable from its arguments.

import subprocess as sp, shlex
sp.Popen(['echo 1']) # FAILS with "No such file or directory"
sp.Popen(['echo', '1']) # SUCCEEDS
sp.Popen(['echo 1'], shell=True) # SUCCEEDS, but extra overhead
sp.Popen(shlex.split('echo 1')) # SUCCEEDS, equivalent to #2

Without shell=True, Popen expects the executable to be the first element of args, which is why it fails, there is no "echo 1" executable. Adding shell=True invokes your system shell and passes the first element of args to the shell. i.e. for linux, Popen(['echo 1'], shell=True) is equivalent to Popen('/bin/sh', '-c', 'echo 1') which is more overhead than you may need. See Popen() documentation for cases when shell=True is actually useful.

edited Jul 19 '17 at 09:28

dǝɥɔS ʇoıןןƎ

1,674
5
19
42

answered May 19 '16 at 20:22

Mark Woodward

928
9
9

This version of the error can occur if you forget to put commas between the string literals, as Python will concatenate them: `sp.Popen(['echo' '1']) # FAILS` – Carl G Jun 19 '17 at 17:08
it should be `import shlex` as shlex is its own [python standard package](https://docs.python.org/3/library/shlex.html). – hard Aug 26 '19 at 15:48
You will even get this error if the command (first arg) has a space in it. For example, this will fail: `sp.Popen(['echo ', '1'])` Notice the space right after "echo"... very easy to miss. – JDune May 16 '20 at 01:27

score 5 · Answer 3 · answered Mar 30 '12 at 04:53

5

You have to give the full path to your program deactivate and then it the subprocess module should be able to find it.

answered Mar 30 '12 at 04:53

Senthil Kumaran

54,681
14
94
131

score 0 · Answer 4 · answered Jun 10 '23 at 08:44

Late to the show because this kind of strange behavior happens in some environment. So I have a setup where I have cmake installed as a portable version. I set the path on a command prompt temporarily so that it doesn't override any existing versions for the user. Strangely , I observe that while typing cmake spawns cmake , calling subprocess popen for cmake doesn't work in THIS environment but WORKS in ALL OTHER setups.. The solutions I found is give full path as is already discussed here or ensure I already set the paths in the Sys environment variable PATH permanently... I had to go with latter due to some DEV constraints.

By the way Shell=true works but I already need the current shell and I pipe the results elsewhere.. so that isn't an option

score 0 · Answer 5 · answered Mar 30 '12 at 09:17

I'm spawning subprocesses like that:

SHUTDOWN_CMD = os.path.sep.join(["c:", "windows", "system32", "shutdown.exe"])

def abortShutdown():
    os.spawnv(os.P_NOWAIT, SHUTDOWN_CMD,
        [SHUTDOWN_CMD, '/A'])
    time.sleep(3)

I'm not using subprocess since Python 2.5 does not support it. I had to use use the FULL path to have it working and I guess you also have to use the full path to your custom commands.

Popen error: "[Errno 2] No such file or directory" when calling shell function

5 Answers5

Linked