1

https://stackoverflow.com/a/12698328/391104

The above post said that shell=True should not be be used within check_output. However, I cannot even make it work without using the shell=True.

What is the issue?

Python 3.5.1

>>> from subprocess import STDOUT, check_output
>>> check_output('ls -l', stderr=STDOUT, timeout=10)

Traceback (most recent call last): File "", line 1, in File "/opt/rh/rh-python35/root/usr/lib64/python3.5/subprocess.py", line 629, in check_output **kwargs).stdout File "/opt/rh/rh-python35/root/usr/lib64/python3.5/subprocess.py", line 696, in run with Popen(*popenargs, **kwargs) as process: File "/opt/rh/rh-python35/root/usr/lib64/python3.5/subprocess.py", line 950, in init restore_signals, start_new_session) File "/opt/rh/rh-python35/root/usr/lib64/python3.5/subprocess.py", line 1544, in _execute_child raise child_exception_type(errno_num, err_msg) FileNotFoundError: [Errno 2] No such file or directory: 'ls -l'

However,

>>> check_output('ls -l', stderr=STDOUT, timeout=10, shell=True)

works as expected

== Update for AnilRedshift ==

# work w/o problems
try:
    subprocess.check_output("rsync -ae 'ssh -q' /tmp/hello*.txt machine:/tmp", timeout=20, shell=True)
except subprocess.TimeoutExpired as e:
    print(e)

# fail
try:
    args = shlex.split("rsync -ae 'ssh -q' /tmp/hello*.txt machine:/tmp")
    subprocess.check_output(args, timeout=20)
except subprocess.TimeoutExpired as e:
    print(e)

CalledProcessError: Command '['rsync', '-ae', 'ssh -q', '/tmp/hello*.txt', 'machine:/tmp']' returned non-zero exit status 23
q0987
  • 34,938
  • 69
  • 242
  • 387

1 Answers1

1

When you use shell=True, the first thing that happens is that your command is parsed via shell semantics.

In this case, ls -l becomes ['ls', '-l'].

So, if you want to do this yourself, without the shell, you need to split your command into arg chunks.

from subprocess import STDOUT, check_output
check_output(['ls', '-l'], stderr=STDOUT, timeout=10)

If you have a more complicated command, you can use shlex.split as a helper:

import shlex
args = shlex.split("ls -l")
check_output(args, stderr=STDOUT, timeout=10)
AnilRedshift
  • 7,937
  • 7
  • 35
  • 59
  • Please see my comment in OP. – q0987 Jul 20 '18 at 20:14
  • 2
    @q0987: Your updated question is a separate issue. `*` has no meaning to `ls`. If you go into a terminal and type `ls *.txt`, your **shell** first parses the command, and expands `'*.txt'` to every .txt in the current directory, and that is passed to ls. As you can see, this glob-expanding is a feature of the shell, and thus why you would need `shell=True` for that to work. – AnilRedshift Jul 20 '18 at 20:20