How do you handle pipes and output in Python?

Question

I'm attempting to execute this command within my python script:

avprobeCommand = "avprobe -of json -show_streams {0} | grep '\"duration\"' | sed -n 1p | sed 's/ //g'".format(hiOutput)
output = subprocess.check_output([avprobeCommand])

and I keep getting:

    Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 552, in __bootstrap_inner
    self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 505, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/jmakaila/Documents/Development/Present/Web/video_dev/present-live-transcoder/Transcoder.py", line 60, in transcode
    output = subprocess.check_output([avprobeCommand])
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 537, in check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1228, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

I've already tried splitting the args up, but I keep getting an error for the -of json -show_streams part, which, for the record, looked like this:

subprocess.check_output(["avprobe", "-of json", "-show_streams", "{0}".format(hiOutput)

Alternative approaches are listed here: http://stackoverflow.com/q/295459/1858225 I recommend using the `pipes` module: https://docs.python.org/2/library/pipes.html — Kyle Strand, Dec 04 '14 at 18:54

falsetru · Accepted Answer · 2013-10-24T11:54:09.107

1

Pass the command as string, and pass shell=True:

import pipes
import subprocess

avprobeCommand = """avprobe -of json -show_streams {0} | grep '"duration"' | sed -n 1p | sed 's/ //g'""".format(pipes.quote(hiOutput))
output = subprocess.check_output(avprobeCommand, shell=True)

UPDATE: argument should be escaped using pipes.quote. (Use shlex.quote if you use Python 3.3+).

edited Oct 24 '13 at 11:54

answered Oct 19 '13 at 03:47

falsetru

357,413
63
732
636

@J.F.Sebastian, Thank you for comment. I updated the answer according to your comment. – falsetru Oct 24 '13 at 11:54

score 1 · Answer 2 · edited May 23 '17 at 12:28

In your case, you could move the post-processing into Python:

import json
from subprocess import check_output as qx

data = json.loads(qx(["avprobe", "-of", "json", "-show_streams", hiOutput]))
result = data["duration"]         # grep '"duration"'
             .partition("\n")[0]  # sed -n 1p
             .replace(" ", "")    # sed 's/ //g'

For a more general case, see How do I use subprocess.Popen to connect multiple processes by pipes?

How do you handle pipes and output in Python?

2 Answers2