0

It seems that Opencv + FFMPEG is corrupting lossless video frames.

First I establish that the video is lossless with this test

gst-launch-1.0.exe {RAW_IMAGE_SOURCE} ! video/x-raw,width=1280,height=480,format=I420 ! x264enc pass=quant quantizer=0 byte-stream=true cabac=false ! video/x-h264, stream-format=byte-stream ! avdec_h264 ! jpegenc quality=100 !  multifilesink location=frames/image_%06d.jpg

The images correctly replicate very small RGB values (16,16,16) in large regions of the background. However, whenever I replace the multifilesink with an udpsink as below

gst-launch-1.0.exe {RAW_IMAGE_SOURCE} ! video/x-raw,width=1280,height=480,format=I420 ! x264enc pass=quant quantizer=0 byte-stream=true ! video/x-h264, stream-format=byte-stream ! rtph264pay name=pay0 pt=96 ! udpsink host=127.0.0.1 port=6666 sync=false

and try to open it with OpenCV + Python with the SDP "h264rtp.sdp" below

c=IN IP4 127.0.0.1
m=video 6666 RTP/AVP 96 
a=rtpmap:96 H264/90000

and this code

url = "h264rtp.sdp"
cap_receive = cv2.VideoCapture(url, cv2.CAP_FFMPEG)

The frames returned by this video capture are different than the images saved by jpegenc. More specifically, the large dark regions go from (16,16,16) and (17,17,17) to (0,0,0)!!!

  • ffmpeg version 4.4.1
  • gst-launch-1.0 version 1.16.3
  • GStreamer 1.16.3

I am required to provide 100% lossless video... any suggestions?

Milardo
  • 37
  • 7
  • I suppose the input is YUV420 (I420) and the output is BGR. The color conversion from YUV to BGR does convert `(16,16,16)` to `(0,0,0)`. Can you make a reproducible example? Post an example for `RAW_IMAGE_SOURCE` (Is it actually in I420 format?) . I don't know GStreamer that well - how did you get the file `h264rtp.sdp`? – Rotem Apr 03 '22 at 16:20
  • I made the SDP by hand while using information from the search query "minimal working h264 SDP": https://stackoverflow.com/questions/20538698/minimum-sdp-for-making-a-h264-rtp-stream – Milardo Apr 04 '22 at 03:05
  • I'll post a reproducible example tomorrow, thanks for the help – Milardo Apr 04 '22 at 03:05
  • @Rotem you were correct, I was assigning it the wrong format. In case you post this as an answer I'll mark it as correct. – Milardo Apr 06 '22 at 00:57

1 Answers1

0

As mentioned by @Rotem, the format I was passing as a CAPS string was wrong. Correcting the format solved the issue.

As he did not post an answer after a certain amount of time I'll simply leave this here.

Milardo
  • 37
  • 7