0

I created a java program to parse mp4 and ts segments. I muxed a ts fragment with ffmpeg into mp4. I'm trying to understand why ffmpeg formed the mdat the way it is.

Here are the original 4 NALU extracted from TS-PES in hex, ordered by start codes (fig.1):

00 00 00 01 |09 e0
00 00 00 01 |67 42 c0 0d 96 56 1e 33 7f e0 08 00 05 be a0 a0 a0 be 00 00 07 d0 00 00 ea 61 28
00 00 00 01 |68 da 8f 20
00 00 00 01 |65 88 80 40 20 7f 3e 21 11 c2 18 78 b0 00 40 81 0e 04 1a 24 05 48 09 84 67 36 61 14 33 8f eb 05 d6 a5 e4 1e 34 d1 f8 65 08 8c 00 f0 00 ...

These look like legitimate NALU according to my parser (fig.2):

fzb:false nri:0x0 nut:9 [[DATA]] : 09 E0
fzb:false nri:0x3 nut:7 seq_parameter_set_rbsp [[DATA]] : 67 42 C0 0D 96 56 1E 33 7F E0 08 00 05 BE A0 A0 A0 BE 00 00 07 D0 00 00 EA 61 28
fzb:false nri:0x3 nut:8 pic_parameter_set_rbsp [[DATA]] : 68 DA 8F 20
fzb:false nri:0x3 nut:5 slice_layer_without_partitioning_rbsp() fmis:0 st:7(I_2) ppsi:0x0 [[DATA]] : 65 88 80 40 20 7F 3E 21 11 C2 18 78 B0 00 40 81 0E 04 1A 24 05 48 09 84 67 36 61 14 33 8F EB 05 D6 A5 E4 1E 34 D1 F8 65 08 8C 00 F0 00

Then my muxed mp4 mdat as chunked out by SampleTableBox contains this hex data, ordered by what look like malformed NALU's (fig.3):

00 00 00 02 |09 e0
00 00 00 1b |67 42 c0 0d 96 56 1e 33 7f e0 08 00 05 be a0 a0 a0 be 00 00 07 d0 00 00 ea 61 28
00 00 00 04 |68 da 8f 20
00 00 1e ac |65 88 80 40 20 7f 3e 21 11 c2 18 78 b0 00 40 81 0e 04 1a 24 05 48 09 84 67 36 61 14 33 8f eb 05 d6 a5 e4 1e 34 d1 f8 65 08 8c 00 f0 00

Note the shorthand reference for the data above from ITU-T H264 7.4.1 as:

fbz = forbidden_zero_bit
nri = nal_ref_idc
nut = nal_unit_type
fmis = first_mb_in_slice
st = slice_type
ppsi = pic_parameter_set_id

Also note the payload is the same.

I assume the mp4 is correct because it works correctly in any player. But I'm not sure what the mdat muxing has done to the NALU start codes. I'm a bit new to this and had a hard time trying to figure out why this would be.

They don't look like NALU's because the start code doesn't end with 01.

It was suggested they are AvcC (ISO/IEC 14996-15 5.2.4.1.1) but the first byte in each data set start with 00's like NALUs and won't pass the configurationVersion test (first byte 01).

I'd like to know how the mdat is spec'd in this case. If someone could point me to the in the right direction or the unrefuted tome that'd be much appreciated. Thanks!

nilgirian
  • 21
  • 1
  • 5

1 Answers1

1

Looks like you are trying to parse MP4 mdat as Annex B format for H.264 which is incorrect because MP4 use AVCC format for H.264. There was good explanation of this formats in Possible Locations for Sequence/Picture Parameter Set(s) for H.264 Stream

Community
  • 1
  • 1
nobody555
  • 2,239
  • 18
  • 18
  • Thank you so much! – nilgirian Nov 06 '16 at 17:57
  • It seems these aren't in AvcC format either :( I'm still looking for an answer. – nilgirian Nov 12 '16 at 06:22
  • Looks like you are mixing up something (probably AVCC-style NALUs and AVCC extradata). That are simple 4-byte NALU-length fields (depends on NALULengthSizeMinusOne value but usually is 4-byte) so no need for any "configurationVersion test (first byte 01)". Don't fix up on AVCC name here because it is not official name and can be called MP4-style NALU encoding. – nobody555 Nov 12 '16 at 09:43
  • Oh! So it seems to me the the NALULengthSizeMinusOne comes from the AVCConfigurationBox and the stuff in mdat is just the parameterSetNALLengths and parameterSetNALUnits.. this indeed seems to be the answer I'm looking for.. thanks for clarifying. Indeed very helpful. – nilgirian Nov 12 '16 at 21:05