0

I have an input RTSP stream that I would like to manipulate on a frame-by-frame basis using openCV. After these changes are applied, I'd like to create a separate RTSP stream from those frames. I'm piping the resulting JPEG images to FFMPEG via STDIN. I need the intermediate frame to be a JPEG.

In other words, I must conform to this pattern: RTSP IN -> Create JPEG as input -> manipulation, JPEG out -> RTSP

The PROBLEM I'm trying to solve deals with a codec at this point. See the last few lines of FFMPEG's output error message.

Here is what I have:

def open_ffmpeg_stream_process():
    args = (
        "ffmpeg -re -stream_loop -1 "
        "-f jpeg_pipe "
        "-s 512x288 "
        "-i pipe:0 "
        "-c:v h264 "
        "-f rtsp "
        "rtsp://localhost:8100/out0"
    ).split()
    return subprocess.Popen(args, stdin=subprocess.PIPE)

video_source = cv2.VideoCapture('rtsp://localhost:9100/in0')

frame_cnt = 0
FRAME_SKIP = 30

ffmpeg_process = open_ffmpeg_stream_process()
while video_source.isOpened():

    frame_cnt += 1
    if frame_cnt % FRAME_SKIP:
      continue
    else:
      frame_cnt = 0

    _, frame = video_source.read()
    _, jpg = cv2.imencode('.jpg', frame)

    # Work on the JPEG occurs here, and the output will be a JPEG

    ffmpeg_process.stdin.write(jpg.astype(np.uint8).tobytes())

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

video_source.release()

Here is FFMPEG's output:

ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: ---- noisy configuration stuff ----
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, jpeg_pipe, from 'pipe:0':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: mjpeg, rgb24(bt470bg/unknown/unknown), 512x288, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
[libx264 @ 0x562a1ffc5840] using SAR=1/1
[libx264 @ 0x562a1ffc5840] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x562a1ffc5840] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 0x562a1ffc5840] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=18 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[tcp @ 0x562a20826c00] Connection to tcp://localhost:8100?timeout=0 failed: Connection refused
Could not write header for output file #0 (incorrect codec parameters ?): Connection refused
Error initializing output stream 0:0 -- 
Conversion failed!

How does one create an H.264, RTSP stream from a series of JPEG frames using FFMPEG.

Notes: The FFMPEG command/subproccess might need unrelated improvements, feel free to comment on my crappy code.

Edit: Oof, I just found out the FFMPEG command doesn't even work stand alone.

Michael Schmidt
  • 396
  • 1
  • 3
  • 15
  • Duplicate of: https://stackoverflow.com/questions/72762198/convert-series-of-images-to-rtsp – Michael Schmidt Jan 25 '23 at 15:40
  • Why are you using `-stream_loop -1`? Why are you converting the images to JPEG instead of writing raw frames to `stdin`? I suggest you to start with something simpler, as writing synthetic images, and encoding a video file instead of RTSP. I posted an [example](https://stackoverflow.com/a/61281547/4926757) for encoding H.265 video. – Rotem Jan 26 '23 at 21:46

0 Answers0