44

How can I reverse the results of a shlex.split? That is, how can I obtain a quoted string that would "resemble that of a Unix shell", given a list of strings I wish quoted?

Update0

I've located a Python bug, and made corresponding feature requests here.

Matt Joiner
  • 112,946
  • 110
  • 377
  • 526
  • 1
    Out of curiosity, why do you need this if subprocess.Popen takes a list for the command? (when shell=False) – tokland Jan 20 '11 at 15:18
  • @tokland: I'm not actually using the output of shlex.split for Popen, I'm parsing a list of paths provided by the user. I allow them to use shell-style splitting. – Matt Joiner Jan 21 '11 at 01:18

6 Answers6

30

We now (3.3) have a shlex.quote function. It’s none other that pipes.quote moved and documented (code using pipes.quote will still work). See http://bugs.python.org/issue9723 for the whole discussion.

subprocess.list2cmdline is a private function that should not be used. It could however be moved to shlex and made officially public. See also http://bugs.python.org/issue1724822.

Daniel Fortunov
  • 43,309
  • 26
  • 81
  • 106
merwok
  • 6,779
  • 1
  • 28
  • 42
  • 1
    What designates `subprocess.list2cmdline()` as private? It is not underscore prefixed, and mentioned in the official documentation! – Daniel Fortunov Mar 02 '16 at 18:18
  • `shlex.quote` wont work correctly for chaining commands. i.e. dummy `cmd = ['pwd', '&&', 'cd']` which yields incorrect command: `"pwd '&&' cd"`. – vedar Jul 05 '16 at 16:27
  • `subprocess.list2cmdline()` seems like different command and it behaves correctly with chaining symbols, although it's intended for Windows, where user can pass list to `Popen` anyways (even with enabled shell - `shell=True`) – vedar Jul 05 '16 at 16:28
  • 2
    Perhaps it would be good to add an actual example of how to use `shlex.quote` to the answer itself? At https://bugs.python.org/issue22454 they suggest using `' '.join(shlex.quote(x) for x in split_command)` should be sufficient. – Matthijs Kooijman Apr 25 '18 at 12:47
20

How about using pipes.quote?

import pipes
strings = ["ls", "/etc/services", "file with spaces"]
" ".join(pipes.quote(s) for s in strings)
# "ls /etc/services 'file with spaces'"

.

tokland
  • 66,169
  • 13
  • 144
  • 170
11

There is a feature request for adding shlex.join(), which would do exactly what you ask. As of now, there does not seem any progress on it, though, mostly as it would mostly just forward to shlex.quote(). In the bug report, a suggested implementation is mentioned:

' '.join(shlex.quote(x) for x in split_command)

See https://bugs.python.org/issue22454

Matthijs Kooijman
  • 2,498
  • 23
  • 30
  • 2
    That's exactly the implementation currently in Python: [3.9/Lib/shlex.py](https://github.com/python/cpython/blob/3.9/Lib/shlex.py) – famzah Oct 28 '20 at 10:26
11

It's shlex.join() in python 3.8

irdkwmnsb
  • 303
  • 2
  • 9
  • For Python 3.7 or earlier, see the [answer](https://stackoverflow.com/a/50022816/198219) by Matthijs Kooijman. – famzah Oct 28 '20 at 10:27
6

subprocess uses subprocess.list2cmdline(). It's not an official public API, but it's mentioned in the subprocess documentation and I think it's pretty safe to use. It's more sophisticated than pipes.open() (for better or worse).

Larry Hastings
  • 2,644
  • 20
  • 11
  • `list2cmdline` is _only used on windows_, so the escaping it performs is only suitable for windows. – Eric Feb 17 '19 at 06:29
0

While shlex.quote is available in Python 3.3 and shlex.join is available in Python 3.8, they will not always serve as a true "reversal" of shlex.split. Observe the following snippet:

import shlex
command = "cd /home && bash -c 'echo $HOME'"
print(shlex.split(command))
# ['cd', '/home', '&&', 'bash', '-c', 'echo $HOME']
print(shlex.join(shlex.split(command)))
# cd /home '&&' bash -c 'echo $HOME'

Notice that after splitting and then joining, the && token now has single quotes around it. If you tried running the command now, you'd get an error: cd: too many arguments

If you use subprocess.list2cmdline() as others have suggested, it works nicer with bash operators like &&:

import subprocess
print(subprocess.list2cmdline(shlex.split(command)))
# cd /home && bash -c "echo $HOME"

However you may notice now that the quotes are now double instead of single. This results in $HOME being expanded by the shell rather than being printed verbatim as if you had used single quotes.

In conclusion, there is no 100% fool-proof way of undoing shlex.split, and you will have to choose the option that best suites your purpose and watch out for edge cases.

Graham Palmer
  • 53
  • 1
  • 6