Escaping quotation marks in python string

Question

I'm using subprocess to call a program within python and I'm passing a string to it, which can contain quotation marks.

This is the piece of code that is giving me troubles

import subprocess
text = subprocess.Popen("""awk 'BEGIN { print "%s"}' | my_program """ % sentence, stdout=subprocess.PIPE, shell=True)

When sentence = "I'm doing this" I get the following error message

/bin/sh: -c: line 0: unexpected EOF while looking for matching `"'
/bin/sh: -c: line 1: syntax error: unexpected end of file

I guess this has to do with the way quotes are escaped in python and linux. Is there a way to fix it?

try `sentence = "I\'m doing this"`. you're confusing `awk` because there's a quote in your quoted awk expression. — Jean-François Fabre, Mar 27 '17 at 09:34

Jean-François Fabre · Accepted Answer · 2017-03-27T11:24:59.310

1

you're confusing awk and underlying shell because there's a quote in your quoted awk expression. First part is equivalent to:

awk 'BEGIN { print "I'm doing this"}'

Which is incorrect, even in pure shell.

Quickfix, escape the quotes in your sentence:

text = subprocess.Popen("""awk 'BEGIN { print "%s"}' | my_program """ % sentence.replace("'","\\'"), stdout=subprocess.PIPE, shell=True)

Proper fix: don't use awk at all just to print something, just feed input to your subprocess:

text = subprocess.Popen(my_program, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
output,error = text.communicate(sentence.encode())

(and you can get rid of the shell=True in the process)

Last point: you seem to have trouble because my_program is some program plus arguments. To pass a command such as aspell -a you can do:

my_program = "aspell -a"

or:

my_program = ['aspell','-a']

but not

my_program = ['aspell -a']

which is probably what you've done here, so Python tries to literally execute the program "aspell -a" instead of splitting into program + argument.

edited Mar 27 '17 at 11:24

answered Mar 27 '17 at 09:36

Thanks so much for your answer, but I get this error when I try to replace `my_program` with what is should be (`"aspell -a"`) `FileNotFoundError: [Errno 2] No such file or directory: 'aspell -a'`. – Brian Mar 27 '17 at 09:41
You want to avoid a shell and run `Popen(['aspell', 'a'])` instead of putting the first argument as string. See also http://stackoverflow.com/questions/3172470/actual-meaning-of-shell-true-in-subprocess/36008455#36008455 – tripleee Mar 27 '17 at 09:42
Incidentally, avoiding these quoting issues is another good reason to avoid `shell=True`. – tripleee Mar 27 '17 at 09:43
@tripleee actually it works when passing a _string_ with spaces in it. Which _doesn't_ work is passing a _list_ of parameters but with incorrect splitting: `['aspell -a']` – Jean-François Fabre Mar 27 '17 at 11:22
Yeah, with the additional caveat that I guess the last one might actually work on Windows. – tripleee Mar 27 '17 at 11:28
`subprocess.check_output(["dir /B"],shell=True)` fails on windows. So no, if that's what you meant. Windows is _evil_ but even evil has standards :) – Jean-François Fabre Mar 27 '17 at 11:30

1 Answers1