0

I want to use ffmpeg to read an RTSP stream, extract frames via a pipe, do some processing on them with Python and afterwards combine the processed frames via another pipe with the original audio. I'm using the subprocess module in Python to execute the ffmpeg command as well as read and write the frames from and to ffmpeg.

Questions:

  1. Is it possible to pipe both stdin and stdout so as to extract the frames and then feed them back in after the processing?
  2. Do i also have to pipe the audio separately and feed it with the processed frames or can i simply copy the audio stream when mapping the output?
MinasCham
  • 1
  • 1
  • 3
  • **1.** Its possible, but you need to execute two instances of FFmpeg sub-processes. Here is an [example](https://stackoverflow.com/questions/60558412/how-to-decode-a-video-memory-file-byte-string-and-step-through-it-frame-by-f) for pipe both stdin and stdout. **2.** I think you can copy the audio stream by mapping it to the output (with or without re-encoding depends on the video codec and file container). – Rotem Jun 04 '21 at 20:52
  • @Rotem thank you for your answer! If i use two instances of ffmpeg subprocesses, is there a way for the audio to be shared/copied between them? or do i have to also pipe the audio somehow? – MinasCham Jun 07 '21 at 14:29
  • I was thinking about getting the audio directly from the RTSP stream, but it requires two RTSP simultaneous connections (it's not efficient, and I never tried it). Splitting the RTSP into audio output stream pipe and video output stream pipe, process the video, and pipe the original audio and modified video as inputs "sounds" complicated (and I never tried it). – Rotem Jun 07 '21 at 14:43
  • @Rotem so two ffmpeg subprocesses with the first one piping the frames to stdout and the second one getting video input via stdin and audio input via the RTSP stream? I also tried using OpenCV for the frames and ffmpeg for the audio and combining the streams with ffmpeg but the streams are not synced. – MinasCham Jun 07 '21 at 15:17
  • If you post your code, I may try it out. Make sure to use a public RTSP, so I can get the same results as yours. – Rotem Jun 07 '21 at 15:33
  • 1
    I managed to make it work by reading the frames via OpenCV and piping them into an FFmpeg subprocess with the RTSP stream as an additional input (so as to copy the audio stream on the output), thank you for your help! – MinasCham Jun 09 '21 at 07:25
  • @MinasCham, Could you please post the subprocess command on how to get two pipe lines to work? Thanks – ttkrpink May 11 '22 at 08:57

0 Answers0