1

I have a series of videos with custom information encoded in the sei message NAL. Is it possible to read that information when decoding using the Nvidia hardware decoder. If it is not supported, should I use FFMPEG compiled with NVENC support instead?

UPDATE: I want to decode the media and read the SEI message. I am streaming live video and including postprocessing info in the sei message. The client has to use that info to apply effects to the decoded media. Decoding the media as quickly as possible is important, and I want to do it in hardware. I assume that the Nvidia decoder must parse the NAL units to decode them. I would like to avoid duplicating work if possible.

cloudraven
  • 2,484
  • 1
  • 24
  • 49
  • SEI messages do not require decoding of the media. Just a filter is needed. – szatmary Apr 20 '18 at 23:49
  • Sure, but I want to decode the media and read the SEI message. I am streaming live video and including postprocessing info in the sei message. The client has to use that info to apply effects to the decoded media. Decoding the media as quickly as possible is important, and I want to do it in hardware. I assume that the Nvidia decoder must parse the NAL units to decode them. I would like to avoid duplicating work if possible. – cloudraven Apr 23 '18 at 16:27
  • 1
    It will not decide SEI, it will just skip over it. You must decide it in software. – szatmary Apr 23 '18 at 16:35
  • 1
    So then the best way is to read the H.264 stream and extract the sei messages myself and then just feed the unmodified stream to the decoder? Is there a library commonly used for parsing NALs available? I haven't found any, but it seems like a common enough task. – cloudraven Apr 23 '18 at 17:10
  • 1
    I wouldn’t say it’s common, but I have a library for reading and writing closed captions to SEI feel free to use it. https://github.com/szatmary/libcaption look at avc.c specifically. I don’t know of any others except the parsers in ffmpeg, or h264bitstream library (which has other issues) – szatmary Apr 23 '18 at 17:15
  • Thanks. That's helpful. I will take a look. I will probably have to support both H264 and HEVC so maybe I will have to use ffmpeg – cloudraven Apr 23 '18 at 17:53
  • 1
    SEI in HEVC is almost identical. Only difference is the nal_unit_header. – szatmary Apr 23 '18 at 19:57
  • So, the only difference is what is described in Section 7.3.1 of both standards? H.265 having in the header f(1),u(6),u(6),u(3) and H.264 having f(1),u(2),u(5)? It is nice that they are so similar. It will be easy to handle both. Thanks – cloudraven Apr 24 '18 at 23:12
  • If you write a summary of the discussion as an answer, I will accept it – cloudraven Apr 24 '18 at 23:13

0 Answers0