3

Section 17.1.1.1. of the documentation states:

If shell is True, the specified command will be executed through the shell. This can be useful if you are using Python primarily for the enhanced control flow it offers over most system shells and still want convenient access to other shell features such as shell pipes, filename wildcards, environment variable expansion, and expansion of ~ to a user’s home directory.

However, on cygwin, output from bash isn't the same as from Python's subprocess, viz:

Bash:

$ ls -ls ~rbarakx
total 0
0 drwxr-xr-x 1 Administrator None 0 Aug 21 17:54 bash
0 drwxr-xr-x 1 Administrator None 0 Jul 11 09:11 python

Python:

>>> subprocess.call(["ls","-ls","~rbarakx"],shell=True)
RCS  mecha.py  print_unicode.py  req.py  requests_results.html  selen.py
0

It looks as if subprocess.call is executing just ls.

Can you suggest why?

My Environment:
Python: Python 2.7.3 (default, Dec 18 2012, 13:50:09) [GCC 4.5.3] on cygwin
cygwin: CYGWIN_NT-6.1-WOW64 ... 1.7.22(0.268/5/3) ... i686 Cygwin
windows: Windows 7 Ultimate

jfs
  • 399,953
  • 195
  • 994
  • 1,670
boardrider
  • 5,882
  • 7
  • 49
  • 86

3 Answers3

4

On Windows, the shell (cmd.exe) doesn't support the expansion of ~ to a user's home directory. Nor wildcard expansion, for that matter. The Python doc is very UNIX/Linux-oriented on this point.

"But wait!" I hear you cry. "I've installed cygwin, I have bash!" Sure you have, but you can hardly expect Python to know that. You're on Windows, so it's going to use the Windows shell. If it didn't, Python scripts that expected the shell to be cmd.exe would be mighty confused.

kindall
  • 178,883
  • 35
  • 278
  • 309
  • 1
    And the workaround is to explicitly use `bash` as the command. – Mark Ransom Nov 21 '13 at 18:53
  • OP uses cygwin's python. `~` is expanded if you [pass to `subprocess.call` correct arguments](http://stackoverflow.com/a/20137299/4279) – jfs Nov 23 '13 at 19:52
3

Interestingly when passing the command as a string instead of a tuple, it seems to work as if you were executing it in bash.

subprocess.call(["ls -ls ~"], shell=True)

If you insist on using tuple, consider the following workaround:

cmd = subprocess.list2cmdline(["ls","-ls","~rbarakx"])
subprocess.call(cmd, shell=True)

or this:

from os.path import expanduser
subprocess.call(["ls","-ls", expanduser("~rbarakx")])
Specur
  • 3,240
  • 4
  • 23
  • 25
  • Thanks Specur. Just a small note - `subprocess.call(["ls -ls","~rbarakx"], shell=True)` actually does not produce the expected results (all others do). – boardrider Nov 23 '13 at 09:47
  • I see... Oh well, I have edited the answer to reflect this. Thanks for informing about this. – Specur Nov 23 '13 at 22:41
  • `list2cmdline()` uses MS C runtime rules for escaping. It is not appropriate in this case (OP runs it in a Unix (cygwin) environment) If you want to create a string suitable for `sh`, you could use `" ".join(map(pipes.quote, args))` (though it is better to pass `args` directly). It doesn't expand `~` – jfs Nov 24 '13 at 02:41
3

It looks as if subprocess.call is executing just ls.

Can you suggest why?

Because on Unix (cygwin) your call is equivalent to:

# WRONG: DON'T DO IT
rc = subprocess.call(["/bin/sh", "-c"] + ["ls","-ls","~rbarakx"])

In other words the argument list is passed to the shell itself i.e., ls doesn't see -ls or ~rbarakx they are considerred /bin/sh arguments.

What you want is:

rc = subprocess.call(["/bin/sh", "-c"] + ["ls -ls ~rbarakx"])

Or using shell=True:

rc = subprocess.call("ls -ls ~rbarakx", shell=True)

Here's the quote from subprocess docs for reference:

On Unix with shell=True, the shell defaults to /bin/sh. If args is a string, the string specifies the command to execute through the shell. This means that the string must be formatted exactly as it would be when typed at the shell prompt. This includes, for example, quoting or backslash escaping filenames with spaces in them. If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself. That is to say, Popen does the equivalent of:

Popen(['/bin/sh', '-c', args[0], args[1], ...])

emphasis is mine

You could also execute the command without any shell:

import os
import subprocess

rc = subprocess.call(["ls", "-ls", os.path.expanduser("~rbarakx")])
jfs
  • 399,953
  • 195
  • 994
  • 1,670
  • Thanks J.F. Sebastian, it's clear now, and indeed both `subprocess.call("ls -ls ~rbarakx", shell=True)` and `subprocess.call(["ls", "-ls", os.path.expanduser("~rbarakx")])` produce the same results as from the shell. – boardrider Nov 23 '13 at 09:35