Python complex string interpolation on shell command

Question

I have a shell command that I would like to execute, with a python script. It's a combination of ffmpeg, grep, gawk and several other ffmpeg options.

The command

ffmpeg -i http://0.0.0.0:8080/stream/video.mjpeg -vcodec copy -map 0 -f segment -segment_time 2 -loglevel 40 -segment_format mp4 capture-%05d.mp4 2>&1 | grep --line-buffered -Eo "segment:.+ended" | gawk -F "'" '{print $2; system("")}'

If you run this command on terminal, it should return a string

capture-00001.mp4

Now, my objective is to run this with subprocess on Python3. Since the command is rather complex with mixed single and double quotes, it needs to be treated correctly otherwise it will complain EOL while scanning string literal, invalid syntax, etc, etc.

I have tried several string formating methods but none work. Below is one of the ways I've tried.

Script

import os
import datetime
import subprocess

first = "segment:.+ended"
second = "'"
third = '{print $2; system("")}'

if __name__ == "__main__":
    fScript = "ffmpeg -i http://0.0.0.0:8080/stream/video.mjpeg -vcodec copy -map 0 -f segment -segment_time 2 -loglevel 40 -segment_format mp4 capture-%05d.mp4 2>&1 | grep --line-buffered -Eo {} | gawk -F {} {}".format(first, second, third)
    try:
        result = subprocess.check_output(fScript, shell=True).decode('utf-8')
        print(result)
    except subprocess.CalledProcessError as e:
        print(e.output)

The problem is that your command is not a single subprocess it's 3 (`ffmpeg` piped into `grep` piped into `awk`). If you want to replicate this with the subprocess module you could run all 3 as separate subprocesses and connect the pipes together. — Tom Dalton, May 02 '18 at 15:35
See https://stackoverflow.com/questions/295459/how-do-i-use-subprocess-popen-to-connect-multiple-processes-by-pipes for more info — Tom Dalton, May 02 '18 at 15:36
Might have already tried it but you could try: `fscript = r"""ffmpeg -i http://0.0.0.0:8080/stream/video.mjpeg -vcodec copy -map 0 -f segment -segment_time 2 -loglevel 40 -segment_format mp4 capture-%05d.mp4 2>&1 | grep --line-buffered -Eo "segment:.+ended" | gawk -F "'" '{print $2; system("")}'"""` — NBlaine, May 02 '18 at 15:37
@NathanBlaine Your script works, but it does not print the result as intended, I'm guessing its the subprocess. Anyway, can you tell me what does the `r` do? — Rex Low, May 02 '18 at 15:46
Yeah so `r` basically means raw string literal (https://stackoverflow.com/questions/2081640/what-exactly-do-u-and-r-string-flags-do-and-what-are-raw-string-literals). I think what might have made that work there was actually the triple `"` on each side of your command though — NBlaine, May 02 '18 at 15:59
@NathanBlaine Thanks for the suggestion, I tried removing `"` on each side but it breaks the literal. I've made it work by using `subprocess.call` instead. Thanks again for your help :) — Rex Low, May 02 '18 at 16:03

score 2 · Accepted Answer · answered May 02 '18 at 16:02

Thanks to the suggestion from @NathanBlaine, the use of marking string literal with r is exceptionally suitable for my use case here. Always happy to learn a new thing every day. :)

Though, I was not able to retrieve the string result with check_output. Instead, subprocess.call managed to return the expected strings. Here's the working code.

import os
import datetime
import subprocess


if __name__ == "__main__":
    fScript = r"""ffmpeg -i http://0.0.0.0:8080/stream/video.mjpeg -vcodec copy -map 0 -f segment -segment_time 2 -loglevel 40 -segment_format mp4 capture-%05d.mp4 2>&1 | grep --line-buffered -Eo "segment:.+ended" | gawk -F "'" '{print $2; system("")}'"""
    try:
        result = subprocess.call(fScript, stderr=subprocess.STDOUT, shell=True).decode('utf-8')
        print(result)
    except subprocess.CalledProcessError as e:
        print(e.output)

Python complex string interpolation on shell command

1 Answers1