Why does calling ffmpeg mess up reading variables?

Question

A few years ago when I was younger, more carefree, and, uh, less cognisant of good practice in writing shell scripts; I wrote a quick-and-dirty script to assist with a task I was facing:

#!/bin/bash                                                                                                         

# autohighlighter which generates highlights from a video clip and file                                             

process_highlight_file() {                                                                                          
        n=0                                                                                                         
        while read -r line; do                                                                                      
                begin=$(echo "$line" | awk '{ print $1 }' )                                                         
                end=$(echo "$line" | awk '{ print $2 }')                                                            
                hilightname=$(echo "$line" | awk '{ print $3 }')                                                    
                printf "Begin highlight called %s at %s, to %s\n" "$hilightname" "$begin" "$end"                   
                echo "$begin $end"                                                                                 
                sleep 2                                                                                             
                echo "ffmpeg -y -ss $begin -i $videofile -to $end -c copy -avoid_negative_ts 1 $hilightname.mkv"    
        ffmpeg -loglevel quiet -hide_banner -y -ss "$begin" -i "$videofile" -to "$end" -c copy -avoid_negative_ts 1 "$hilightname.mkv"                                                                                                                                                                   
                if [ "$n" -eq 0 ]; then                                                                             
                        echo -n "melt $hilightname.mkv " > constructed.melt                                         
                else                                                                                                
                        echo -n "$hilightname.mkv -mix 120 -mixer luma " >> constructed.melt                        
                fi                                                                                                  
                (( n++ ))                                                                                           
        done < $highlightfile                                                                                       
        echo -n "-consumer avformat:$videofile-highlights.mkv crf=18" >> constructed.melt                           
}                                                                                                                   

highlightfile=$2                                                                                                    
videofile=$1                                                                                                        
process_highlight_file                                                                                              
exit 0

I call it with the video file name and a highlight file with the following tab-separated contents:

3:55    4:15    tutorialcomplete
10:50   11:15   firstkill
13:30   14:00   pickpocket

If I comment out the actual call to ffmpeg, I get sensible output:

Begin highlight called tutorialcomplete at 3:55, to 4:15
3:55 4:15
ffmpeg -y -ss 3:55 -i 2019-08-27 20-31-27.mkv -to 4:15 -c copy -avoid_negative_ts 1 tutorialcomplete.mkv
Begin highlight called firstkill at 10:50, to 11:15
10:50 11:15
ffmpeg -y -ss 10:50 -i /tmp/footage/gamefootage/pending/sor/2019-08-27 20-31-27.mkv -to 11:15 -c copy -avoid_negative_ts 1 firstkill.mkv
Begin highlight called pickpocket at 13:30, to 14:00
13:30 14:00
ffmpeg -y -ss 13:30 -i /tmp/footage/gamefootage/pending/sor/2019-08-27 20-31-27.mkv -to 14:00 -c copy -avoid_negative_ts 1 pickpocket.mkv

All good, if rather overkill on the debug output.

If I uncomment the call to ffmpeg, I get:

Begin highlight called tutorialcomplete at 3:55, to 4:15
3:55 4:15
ffmpeg -y -ss 3:55 -i /tmp/footage/gamefootage/pending/sor/2019-08-27 20-31-27.mkv -to 4:15 -c copy -avoid_negative_ts 1 tutorialcomplete.mkv
Begin highlight called  at firstkill, to 
firstkill 
ffmpeg -y -ss firstkill -i /tmp/footage/gamefootage/pending/sor/2019-08-27 20-31-27.mkv -to  -c copy -avoid_negative_ts 1 .mkv
Begin highlight called pickpocket at 13:30, to 14:00
13:30 14:00
ffmpeg -y -ss 13:30 -i /tmp/footage/gamefootage/pending/sor/2019-08-27 20-31-27.mkv -to 14:00 -c copy -avoid_negative_ts 1 pickpocket.mkv

Naturally, ffmpeg complains that "firstkill" is not a valid time to seek to. If I add further lines to the file I am passing in, it seems to only affect the second pass through the loop. Additional output is also produced:

Enter command: <target>|all <time>|-1 <command>[ <argument>]

The going theory is that a line isn't properly terminated. However, I can't seem to track that down in the script or the input.

I'm aware there are a number of poor practices, foibles and other unexpected behaviour here, for which past-me definitely takes the blame and hangs their head in shame! That said, why does calling ffmpeg in the loop here cause values from a file to be parsed incorrectly, or: why does commenting out the line with the call to ffmpeg give the variables the correct value. Also, why does it only affect the second pass of the loop? Where does Enter command come from?

Additionally, shellcheck.net doesn't complain about anything in the code.

FFMpeg is probably consuming the stdin. You want to prevent it from doing that with the `-nostdin` option or ` — Léa Gris, Aug 28 '19 at 14:17
Possible duplicate of [execute ffmpeg command in a loop](https://stackoverflow.com/questions/21634088/execute-ffmpeg-command-in-a-loop) — oguz ismail, Aug 28 '19 at 14:19

score 5 · Accepted Answer · edited Jun 20 '20 at 09:12

You need to prevent FFMpeg from consuming the stdin characters stream.

See: man ffmpeg.1

-stdin

Enable interaction on standard input. On by default unless standard input is used as an input. To explicitly disable interaction you need to specify -nostdin.

Disabling interaction on standard input is useful, for example, if ffmpeg is in the background process group. Roughly the same result can be achieved with ffmpeg ... < /dev/null but it requires a shell.

Here in your loop, ffmpeg is consuming the input from your while loop commands block.

while read -r line; do                                                                                      
...
  # here ffmpeg defaults to consuming the same input
  # $highlightfile that is fed to the while loop commands block.
  ffmpeg -loglevel quiet -hide_banner -y -ss "$begin" -i "$videofile" -to "$end" -c copy -avoid_negative_ts 1 "$hilightname.mkv"                                                                                                                                                                   
done < $highlightfile

A fixed version of your code:

#!/usr/bin/env bash

# autohighlighter which generates highlights from a video clip and file

process_highlight_file() {
  videofile="$1"
  highlightfile="$2"

  n=0
  melt_caller=(melt)

  while read -r begin end hilightname; do
    printf 'Begin highlight called %s at %s, to %s\n' "$hilightname" "$begin" "$end"
    ffmpeg_caller=(ffmpeg -nostdin -loglevel quiet -hide_banner -y -ss "$begin" -i "$videofile" -to "$end" -c copy -avoid_negative_ts 1 "$hilightname.mkv")
    env printf '%q ' "${ffmpeg_caller[@]}" && echo
    "${ffmpeg_caller[@]}"
    ((n++)) && melt_caller+=(-mix 120 -mixer luma)
    melt_caller+=("$hilightname.mkv")
  done <"$highlightfile"
  melt_caller+=(-consumer "avformat:$videofile-highlights.mkv" 'crf=18')
  env printf '%q ' "${melt_caller[@]}" && echo
  "${melt_caller[@]}"
}

process_highlight_file "$@"

You have the `while read ...; do` and `done < $highlightfile` at the end of the loop. _In_ the loop you also call `ffmpeg` which also reads from stdin (which is the `$highlightfile`). — Alfe, Aug 28 '19 at 14:29
@Alfe I guess past-me needs to read up on how input redirection works with loops! (right?) — bertieb, Aug 28 '19 at 14:30
Next time instead of answering a well-known duplicate, flag it as duplicate. — oguz ismail, Aug 28 '19 at 14:44
@oguzismail Is there a better duplicate candidate? The accepted answer in that question does not address the issue at hand here, and the most-upvoted answer on the dupe gives the fix but does not explain the reason why it is needed, which I think is pertinent to an answer. Alternatively, editing the upvoted answer on the proposed dupe to give an explanation would be straightforward and not in conflict with the author's intent. Either of these may make it easier for future querents to find the already-extant answer and prevent future dupes :) — bertieb, Aug 28 '19 at 15:00

Why does calling ffmpeg mess up reading variables?

1 Answers1