I have an input RTSP stream that I would like to manipulate on a frame-by-frame basis using openCV. After these changes are applied, I'd like to create a separate RTSP stream from those frames. I'm piping the resulting JPEG images to FFMPEG via STDIN. I need the intermediate frame to be a JPEG.
In other words, I must conform to this pattern: RTSP IN -> Create JPEG as input -> manipulation, JPEG out -> RTSP
The PROBLEM I'm trying to solve deals with a codec at this point. See the last few lines of FFMPEG's output error message.
Here is what I have:
def open_ffmpeg_stream_process():
args = (
"ffmpeg -re -stream_loop -1 "
"-f jpeg_pipe "
"-s 512x288 "
"-i pipe:0 "
"-c:v h264 "
"-f rtsp "
"rtsp://localhost:8100/out0"
).split()
return subprocess.Popen(args, stdin=subprocess.PIPE)
video_source = cv2.VideoCapture('rtsp://localhost:9100/in0')
frame_cnt = 0
FRAME_SKIP = 30
ffmpeg_process = open_ffmpeg_stream_process()
while video_source.isOpened():
frame_cnt += 1
if frame_cnt % FRAME_SKIP:
continue
else:
frame_cnt = 0
_, frame = video_source.read()
_, jpg = cv2.imencode('.jpg', frame)
# Work on the JPEG occurs here, and the output will be a JPEG
ffmpeg_process.stdin.write(jpg.astype(np.uint8).tobytes())
if cv2.waitKey(1) & 0xFF == ord('q'):
break
video_source.release()
Here is FFMPEG's output:
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
configuration: ---- noisy configuration stuff ----
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Input #0, jpeg_pipe, from 'pipe:0':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: mjpeg, rgb24(bt470bg/unknown/unknown), 512x288, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
[libx264 @ 0x562a1ffc5840] using SAR=1/1
[libx264 @ 0x562a1ffc5840] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x562a1ffc5840] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 0x562a1ffc5840] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=18 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[tcp @ 0x562a20826c00] Connection to tcp://localhost:8100?timeout=0 failed: Connection refused
Could not write header for output file #0 (incorrect codec parameters ?): Connection refused
Error initializing output stream 0:0 --
Conversion failed!
How does one create an H.264, RTSP stream from a series of JPEG frames using FFMPEG.
Notes: The FFMPEG command/subproccess might need unrelated improvements, feel free to comment on my crappy code.
Edit: Oof, I just found out the FFMPEG command doesn't even work stand alone.