2

I need to check the first frame in H264 stream is I-Frame.

On the input I have fragmented mp4 file. I tried to find type of frame in the moof->traf->trun "Sample depends on" flag. But seems, not every container has this flag filled in. So I want to try determine type of frame in the mdat block with raw data.

I need to check only the first frame is I-frame in the each fragment. Information about other frames doesn't matter.

How I can do it?

Валентин Никин
  • 357
  • 2
  • 11

1 Answers1

3

You can check the NAL unit type. NAL unit type 5 indicates an IDR frame which is an I frame. Inside the 'mdat' the video is stored:

<size><NAL><size><NAL>...<size><NAL>

The lower 5 bits of the first byte of each NAL unit indicates the type. Skip through types 6,7,8 and 9 until you find type 1 (non IDR frame) or type 5 (IDR frame).

MP4 files should not contain start codes ([00] 00 00 01) or access unit delimiters.

MPEG-2 Transport Streams or *.h264 raw contain start codes ([00] 00 00 01) and access code delimiters.

The size field in MP4 is most of the time 4 bytes but if you want the correct answer you have parse the codec private data (SPS/PPS).

In short H.264 comes in two formats:

  • Annex-B or
  • MP4 (mdat)

Annex-B (MPEG-2 TS, or *.264 raw file):

<[00] 00 00 01> <NAL> <[00] 00 00 01> <NAL> ... <[00] 00 00 01> <NAL>

MP4 (mdat):

<size><NAL><size><NAL>...<size><NAL>

Your file in https://drive.google.com/file/d/1Vwcz8WsTuRLJie8SFzGspizyTc-caGjc/view?usp=sharing has video and audio in the same mdat.

So to get the I-frame detection reliable you have to parse a little more:

this gives you the video start into mdat:

moof[i]->traf[0]->trun[0]->dataOffset

audio starts here => stop parsing video

moof[i]->traf[1]->trun[0]->dataOffset
Markus Schumann
  • 7,636
  • 1
  • 21
  • 27
  • Do you know how much bytes take ? – Валентин Никин Jul 24 '21 at 10:55
  • Are you sure what in mdat block stored the raw h264? NAL unit must start with "00 00 00 01" or "00 00 01" bytes. I reviewed mdat block with hex editor, and I doesn't found any NAL block. Also I tried to convert mp4 file to raw .h264 file with ffmpeg, and I got different data with mdat block. – Валентин Никин Jul 24 '21 at 11:41
  • @ВалентинНикин - I updated my answer and if you share your file - I'll take a look. – Markus Schumann Jul 26 '21 at 11:39
  • Hello, Markus! Thank you. You can take my file from this link. It's fmp4 segment https://drive.google.com/file/d/1Vwcz8WsTuRLJie8SFzGspizyTc-caGjc/view?usp=sharing – Валентин Никин Jul 26 '21 at 13:13
  • hello again. I tried to seek by 844 bytes from the mdat block start point *000048 Track Fragment Run (176 bytes) 000048 Header (8 bytes) ... 000054 sample_count: 13 (0x0000000D) 000058 data_offset: **844** (0x0000034C) 00005C sample (12 bytes) ...* But I got the biggest NAL Size "64 4A 55 D6". Is it right? – Валентин Никин Jul 26 '21 at 20:20
  • The offset is relative to the enclosing moof box. So you seek only 8 bytes into the 'mdat' – Markus Schumann Jul 26 '21 at 21:23