Passing double quote shell commands in python to subprocess.Popen()?

Question

I've been trying to pass a command that works only with literal double quotes in the commandline around the "concat:file1|file2" argument for ffmpeg.

I cant however make this work from python with subprocess.Popen(). Anyone have an idea how one passes quotes into subprocess.Popen?

Here is the code:

command = "ffmpeg -i "concat:1.ts|2.ts" -vcodec copy -acodec copy temp.mp4"

output,error = subprocess.Popen(command, universal_newlines=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE).communicate()

When I do this, ffmpeg won't take it any other way other than quotes around the concat segement. Is there a way to successfully pass this line to subprocess.Popen command?

Answer 1

I'd suggest using the list form of invocation rather than the quoted string version:

command = ["ffmpeg", "-i", "concat:1.ts|2.ts", "-vcodec", "copy",
           "-acodec", "copy", "temp.mp4"]
output,error  = subprocess.Popen(
                    command, universal_newlines=True,
                    stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()

This more accurately represents the exact set of parameters that are going to be passed to the end process and eliminates the need to mess around with shell quoting.

That said, if you absolutely want to use the plain string version, just use different quotes (and shell=True):

command = 'ffmpeg -i "concat:1.ts|2.ts" -vcodec copy -acodec copy temp.mp4'
output,error  = subprocess.Popen(
                    command, universal_newlines=True, shell=True,
                    stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()

Answer 2

Either use single quotes 'around the "whole pattern"' to automatically escape the doubles or explicitly "escape the \"double quotes\"". Your problem has nothing to do with Popen as such.

Just for the record, I had a problem particularly with a list-based command passed to Popen that would not preserve proper double quotes around a glob pattern (i.e. what was suggested in the accepted answer) under Windows. Joining the list into a string with ' '.join(cmd) before passing it to Popen solved the problem.

Answer 3

This works with python 2.7.3 The command to pipe stderr to stdout has changed since older versions of python:

Put this in a file called test.py:

#!/usr/bin/python
import subprocess

command = 'php -r "echo gethostname();"'
p = subprocess.Popen(command, universal_newlines=True, shell=True, 
stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
text = p.stdout.read()
retcode = p.wait()
print text

Invoke it:

python test.py

It prints my hostname, which is apollo:

apollo

Read up on the manual for subprocess: http://docs.python.org/2/library/subprocess.html

Answer 4

I have been working with a similar issue, with running a relatively complex command over ssh. It also had multiple double quotes and single quotes. Because I was piping the command through python, ssh, powershell etc.

If you can instead just convert the command into a shell script, and run the shell script through subprocess.call/Popen/run, these issues will go away.

So depending on whether you are on windows or on linux or mac, put the following in a shell script either (script.sh or script.bat)

ffmpeg -i "concat:1.ts|2.ts" -vcodec copy -acodec copy temp.mp4

Then you can run

import subprocess; subprocess.call(`./script.sh`; shell=True)

Without having to worry about single quotes, etc.

Answer 5

This line of code in your question isn't valid Python syntax:

command = "ffmpeg -i "concat:1.ts|2.ts" -vcodec copy -acodec copy temp.mp4"

If you had a Python file with just this line in it, you would get a syntax error. A string literal surrounded with double quotes can't have double quotes in them unless they are escaped with a backslash. So you could fix that line by replacing it with:

command = "ffmpeg -i \"concat:1.ts|2.ts\" -vcodec copy -acodec copy temp.mp4"

Another way to fix this line is to use single quotes for the string literal in Python, that way Python is not confused when the string itself contains a double quote:

command = 'ffmpeg -i "concat:1.ts|2.ts" -vcodec copy -acodec copy temp.mp4'

Once you have fixed the syntax error, you can then tackle the issue with using subprocess, as explained in this answer. I also wrote this answer to explain a helpful mental model for subprocess in general.

Answer 6

Also struggling with a string argument containing spaces and not wanting to use the shell=True.

The solution was to use double quotes for the inside strings.

args = ['salt', '-G', 'environment:DEV', 'grains.setvals', '{"man_version": "man-dev-2.3"}']

try:
    p = subprocess.Popen(args, stdin=subprocess.PIPE
                             , stdout=subprocess.PIPE
                             , stderr=subprocess.PIPE
                        )
    (stdin,stderr) = p.communicate()
except (subprocess.CalledProcessError, OSError ) as err:
    exit(1)
if p.returncode != 0:
    print("Failure in returncode of command:")

Answer 7

Anybody suffering from this pain. It also works with params enclosed with quotation marks.

params = ["ls", "-la"]
subprocess.check_output(" ".join(params), shell=True)