ffmpeg-python tracking transcoding process

Question

My question is based on https://github.com/kkroening/ffmpeg-python/blob/master/examples/show_progress.py Ideally, all I want is to keep track e.g 0 - 100 for the transcoding process, emitting a streaming yield response to my gRPC client. Technically, I do not need a progress bar. How can I provide my own socket to the ffmpeg subprocess and handle write events to it?

Answer 1

You can redirect "progress" to stdout using -progress pipe:1.
See How to redirect -progress option output of ffmpeg to stderr?

The hard part is to actually get the progress in percentage.

• Start by counting the total number of frames using FFprobe as described here.
• Execute FFmpeg as sub-process, redirect -progress to stdout.
• Start a thread that reads text lines from stdout.
  The thread looks for frame=xx, get the frame, and put it in a list (list of 1 element).
• Execute a "main loop" for demonstrating the progress readings.
  The loop sleeps 1 second, reads the last element from the queue, and prints the progress.

The code starts by building a synthetic video file input.mp4 - used as input.

Here is a "self contained" code sample:

import subprocess as sp
import shlex
import json
from threading import Thread
import time


def progress_reader(procs, q):
    while True:
        if procs.poll() is not None:
            break  # Break if FFmpeg sun-process is closed

        progress_text = procs.stdout.readline()  # Read line from the pipe

        # Break the loop if progress_text is None (when pipe is closed).
        if progress_text is None:
            break

        progress_text = progress_text.decode("utf-8")  # Convert bytes array to strings

        # Look for "frame=xx"
        if progress_text.startswith("frame="):
            frame = int(progress_text.partition('=')[-1])  # Get the frame number
            q[0] = frame  # Store the last sample


# Build synthetic video for testing:
################################################################################
sp.run(shlex.split('ffmpeg -y -f lavfi -i testsrc=size=320x240:rate=30 -f lavfi -i sine=frequency=400 -f lavfi -i sine=frequency=1000 -filter_complex amerge -vcodec libx265 -crf 17 -pix_fmt yuv420p -acodec aac -ar 22050 -t 30 input.mp4'))
################################################################################


# Use FFprobe for counting the total number of frames
################################################################################
# Execute ffprobe (to show streams), and get the output in JSON format
# Actually counts packets instead of frames but it is much faster
# https://stackoverflow.com/questions/2017843/fetch-frame-count-with-ffmpeg/28376817#28376817
data = sp.run(shlex.split('ffprobe -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 -of json input.mp4'), stdout=sp.PIPE).stdout
dict = json.loads(data)  # Convert data from JSON string to dictionary
tot_n_frames = float(dict['streams'][0]['nb_read_packets'])  # Get the total number of frames.
################################################################################

# Execute FFmpeg as sub-process with stdout as a pipe
# Redirect progress to stdout using -progress pipe:1 arguments
process = sp.Popen(shlex.split('ffmpeg -y -loglevel error -i input.mp4 -acodec libvorbis -vcodec libvpx-vp9 -crf 20 -pix_fmt yuv420p -progress pipe:1 output.webm'), stdout=sp.PIPE)

q = [0]  # We don't really need to use a Queue - use a list of of size 1
progress_reader_thread = Thread(target=progress_reader, args=(process, q))  # Initialize progress reader thread
progress_reader_thread.start()  # Start the thread

while True:
    if process.poll() is not None:
        break  # Break if FFmpeg sun-process is closed

    time.sleep(1)  # Sleep 1 second (do some work...)

    n_frame = q[0]  # Read last element from progress_reader - current encoded frame
    progress_percent = (n_frame/tot_n_frames)*100   # Convert to percentage.
    print(f'Progress [%]: {progress_percent:.2f}')  # Print the progress


process.stdout.close()          # Close stdin pipe.
progress_reader_thread.join()   # Join thread
process.wait()                  # Wait for FFmpeg sub-process to finish

---

Note:

• The code sample assumes that ffmpeg and ffprobe are in the executable path.

---

Sample output:

Progress [%]: 7.33
Progress [%]: 16.00
Progress [%]: 24.67
Progress [%]: 33.33
Progress [%]: 42.13
Progress [%]: 50.40
Progress [%]: 58.80
Progress [%]: 67.20
Progress [%]: 75.60
Progress [%]: 84.00
Progress [%]: 92.40
Progress [%]: 100.00