Bash process substitution in Python with Popen

Question

I'm attempting to create a looped video file by calling ffmpeg from the python subprocess library. Here's the part that's giving me problems:

import subprocess as sp
sp.Popen(['ffmpeg', '-f', 'concat', '-i', "<(for f in ~/Desktop/*.mp4; do echo \"file \'$f\'\"; done)", "-c", "copy", "~/Desktop/sample3.mp4"])

With the above code I'm getting the following error:

<(for f in /home/delta/Desktop/*.mp4; do echo "file '$f'"; done): No such file or directory

I did find a similarly phrased question here. But I'm not sure how the solution might apply to solving my issue.

You are running `ffmpeg`, but redirection is a feature of a shell. Use `shell=True` parameter to force `Popen` to pass it through the shell. Also consider not passing an array of command and parameters, but a single string. Read more [here](https://docs.python.org/2/library/subprocess.html#popen-constructor) — Amadan, Oct 08 '14 at 02:10
If you use `shell=True`, the parameters need to passed as a string, not a list. — dano, Oct 08 '14 at 02:11
+1 to what Amadan and dano said. You will likely also need to explicitly indicate that you want `/bin/bash` instead of the default `/bin/sh` for that `Popen`. — Etan Reisner, Oct 08 '14 at 02:12
See the discussion on http://stackoverflow.com/a/24560058/258523 for why a single string argument to `Popen` with `shell=True`. — Etan Reisner, Oct 08 '14 at 02:17
@moorej, re: the edit removing the answer from the question, see [What is the appropriate action when the answer to a question is added to the question itself?](https://meta.stackoverflow.com/questions/267434/what-is-the-appropriate-action-when-the-answer-to-a-question-is-added-to-the-que) -- answers belong _only_ in the answer section, never in the text of a question. — Charles Duffy, Jul 08 '22 at 16:49
you can implement the process substitution in Python but it may be less readable than corresponding bash solution e.g. https://stackoverflow.com/questions/28840575/multiple-pipes-in-subprocess — jfs, Sep 07 '22 at 18:44

score 4 · Accepted Answer · edited Sep 07 '22 at 13:51

Following the advice in the comments and looking elsewhere I ended up changing the code to this:

sp.Popen("ffmpeg -f concat -i <(for f in ~/Desktop/*.mp4; do echo \"file \'$f\'\"; done) -c copy ~/Desktop/sample3.mp4",
         shell=True, executable="/bin/bash")

--which works fine. – moorej

If you need to parameterize input and output files, consider breaking out your parameters:

# sample variables
inputDirectory = os.path.expanduser('~/Desktop')
outputDirectory = os.path.expanduser('~/dest.mp4')

sp.Popen(['''ffmpef -f concat -i <(for f in "$1"/*; do
                                     echo "file '$f'";
                                   done) -c copy "$2" ''',
          bash,            # this becomes $0
          inputDirectory,  # this becomes $1
          outputDirectory, # this becomes $2
         ], shell=True, executable="/bin/bash")

...as this ensures that your code won't do untoward things even when given an input directory with a hostile name like /uploads/$(rm -rf ~)'$(rm -rf ~)'. (ffmpeg is likely to fail to parse an input file with such a name, and if there's any video you don't want included in the current working directory but we'd need to know the escaping rules it uses to avoid that; but it's far better for ffmpeg to fail than to execute arbitrary code).

A short explanation why `bin/bash` helps to process substitution `<()` would be nice — Fibo Kowalsky, Aug 01 '18 at 12:30
@Fibo Kowalsky - It's simply because [process substitution](https://www.gnu.org/software/bash/manual/html_node/Process-Substitution.html) is a special feature of `bash`, which other shells don't have. — Armali, Aug 01 '18 at 13:13

Bash process substitution in Python with Popen

1 Answers1

Linked