escape a string for shell commands in Python

Question

I'm interested to escape a string in Python3.x, such as:

SOME_MACRO(a, b)

into...

SOME_MACRO\(a,\ b\)

... so that it can be passed to a program (not gcc in this case) as a define,

eg,

some_program -DSOME_MACRO\(a,\ b\)="some expression"

I would expect shlex would have this functionality but I didn't find how to do this and checked many similar questions already.

I don't mind to write some simple function to do this, but this seems the kind of thing Python would include.

Note: the program I'm passing the argument to wont accept:

-D"SOME_MACRO(a, b)"="some expression"

... it expects the first character to be an identifier.

score 13 · Answer 1 · answered Aug 08 '13 at 02:54

In Python 3.3 you can use shlex.quote to return a shell-escaped version of a string. It is the successor of pipes.quote, which was deprecated since Python 1.6. Note that the documentation recommends this for cases where you cannot use a list, as suggested in another answer. Also according to the documentation, the quoting is compatible with UNIX shells. I can't guarantee that it will work for your case, but a quick test with rm, using pipes because I don't have Python 3.3:

$ touch \(a\ b\)
$ ls
(a b)

>>> import subprocess, pipes
>>> filename = pipes.quote("(a b)")
>>> command = 'rm {}'.format(filename)
>>> subprocess.Popen(command, shell=True)

$ ls
$

This is awesome! It's not everyday that I learn something new about Python ;) — Chris Cogdon, Nov 25 '17 at 21:15
Also, for python2/3 compatibility: `from six.moves import shlex_quote` — Chris Cogdon, Nov 25 '17 at 22:07

score 2 · Answer 2 · answered Aug 08 '13 at 01:35

2

Doing it correctly means not having to worry about this. The shell has to worry about spaces and quotes and parens; Python does not.

proc = subprocess.Popen([..., "-DSOME_MACRO(a, b)=some expression", ...], ...)

answered Aug 08 '13 at 01:35

Ignacio Vazquez-Abrams

776,304
153
1,341
1,358

2

A bit more explanation of why you don't have to worry about it may be warranted. – user2357112 Aug 08 '13 at 01:37
You are pointing out that you don't always need to escape the string, this is fine and I knew that, but escaping the string remains a valid question (IMHO), you may be writing a shell script and not running the commands directly for example. – ideasman42 Aug 08 '13 at 02:15
If you're writing a shell script then it's not a Python question. – Ignacio Vazquez-Abrams Aug 08 '13 at 02:16
@ideasman42, did you see shlex.quote() on Python 3.3? It's the successor of pipes.quote() to escape shell strings. – Paulo Almeida Aug 08 '13 at 02:22
@Paulo Almeida, yes, but this is quite different escaping spaces for example doesn't work the same way, `shlex.quote("a b") --> "'a b'"` not `"a\ b"` – ideasman42 Aug 08 '13 at 02:25
1

Both methods are the same to the shell. – Ignacio Vazquez-Abrams Aug 08 '13 at 02:26
2

@Ignacio Vazquez-Abrams, By "Writing a shell script", I mean generating a shell script from Python. (generating commands to run generally) – ideasman42 Aug 19 '13 at 07:19
So then why are you using `subprocess`? – Ignacio Vazquez-Abrams Aug 19 '13 at 08:10
1

@Ignacio Vazquez-Abrams, there is no reference to `subprocess` in my question. – ideasman42 Sep 28 '13 at 10:13
Not applicable to Python 2.7.6. With `shell=True`, that version expects that the command be given as a string, hence it expects it to be pre-formatted. If given as an array, `Popen` will use only the first argument in the array. Pretty crappy and undocumented behavior. – Mihai Danila Apr 27 '15 at 20:03

escape a string for shell commands in Python

2 Answers2

Linked