How to fix corrupted video (green frames) after muxing using MediaMuxer and MediaCodec

Question

I am trying to encode an mp4 video from an array of NalUnits where each unit is a frame represented by byte[] that is saved from an rtp packet from an rtsp stream. I am encoding 451 frames at a 30fps where the first frame data is the spp and pps frame combined. This is my current configuration:

Width: 720
Height: 480
Bitrate: 10_000_000
Fps: 30
Colorformat: COLOR_FormatYUV420Flexible
Key I Frame Interval: 1

Here is my method where I encode the configure the encoder:

public boolean createMp4Video(FrameQueue.Frame[] frames, File dir) {
    try {
        // Set up MediaMuxer
        Date date = new Date();
        File file = new File(dir, date + "-recording.mp4");
        this.muxer = new MediaMuxer(file.getAbsolutePath(), MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);

        // Set up MediaCodec
        mMediaFormat = MediaFormat.createVideoFormat(codecName, width, height);
        mMediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, colorFormat);
        mMediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, bitrate);
        mMediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, framerate);
        mMediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1);

        codec = MediaCodec.createEncoderByType(codecName);
        codec.configure(mMediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);

        codec.start();

        MediaCodec.BufferInfo videoBufferInfo = new MediaCodec.BufferInfo();

        // Process frames
        for (int i = 0; i < frames.length; i++) {
            if (i == frames.length - 1) { // if last frame, add end of stream flag
                muxFrame(frames[i].getData(), i, true);
            } else {
                muxFrame(frames[i].getData()), i, false);                     
        }

            // Stop and release resources
            muxer.stop();
            muxer.release();
            codec.stop();
            codec.release();
            listener.onMp4VideoCreated();
            return true;
        } catch (IOException e) {
            return false;
        }
    }

I loop through all the frames using a simple for loop and call the following function to method:

private void muxFrame(byte[] frameData, int encodeIndex, boolean isVideoEOS) {
    long presentationTimeUs = 1000000 / framerate * encodeIndex

    int inputBufferIndex = codec.dequeueInputBuffer(-1);
    if (inputBufferIndex >= 0) {
        ByteBuffer inputBuffer = codec.getInputBuffer(inputBufferIndex);
        inputBuffer.put(frameData);
        codec.queueInputBuffer(inputBufferIndex, 0, frameData.length, presentationTimeUs, 0);
    }

    // Create a MediaCodec.BufferInfo to hold the frame information
    MediaCodec.BufferInfo muxerOutputBufferInfo = new MediaCodec.BufferInfo();

    int outputBufferIndex = codec.dequeueOutputBuffer(muxerOutputBufferInfo, 1000);

    switch (outputBufferIndex) {
        case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
            MediaFormat newFormat = codec.getOutputFormat();
            mVideoTrack = muxer.addTrack(newFormat);
            muxer.start();
            break;
        case MediaCodec.INFO_TRY_AGAIN_LATER:
            break;
        default:
            // Set the size, presentation time, and flags of the muxerOutputBufferInfo
            muxerOutputBufferInfo.size = frameData.length;
            muxerOutputBufferInfo.offset = 0;
            muxerOutputBufferInfo.flags = isVideoEOS ? MediaCodec.BUFFER_FLAG_END_OF_STREAM : MediaCodec.BUFFER_FLAG_KEY_FRAME;
            muxerOutputBufferInfo.presentationTimeUs = presentationTimeUs;
            // Write the frame data to the muxer
            muxer.writeSampleData(mVideoTrack, ByteBuffer.wrap(frameData), muxerOutputBufferInfo);
            codec.releaseOutputBuffer(outputBufferIndex, false);
            break;
    }


}

I am able to save the video and play it but all the frames are corrupted and green as such https://dropmefiles.com/rn6I0

Here is my log when I am encoding:

I/OMXClient: IOmx service obtained
W/OMXUtils: do not know color format 0x7f000200 = 2130706944
W/OMXUtils: do not know color format 0x7f000789 = 2130708361
I/ACodec: [OMX.MTK.VIDEO.ENCODER.AVC] using color format 0x7f420888 in place of 0x7f420888
I/ACodec_vilte: set I frame rate
I/ACodec_vilte: set framerate
I/ACodec_vilte: setMTKParameters, width: 720
I/ACodec_vilte: setMTKParameters, height: 480
I/ACodec: setupAVCEncoderParameters with [profile: Baseline] [level: Level41]
I/ACodec: [OMX.MTK.VIDEO.ENCODER.AVC] cannot encode color aspects. Ignoring.
I/ACodec: [OMX.MTK.VIDEO.ENCODER.AVC] cannot encode HDR static metadata. Ignoring.
I/ACodec: setupVideoEncoder succeeded
I/ACodec_vilte: set I frame rate
I/ACodec_vilte: set framerate
I/ACodec_vilte: setMTKParameters, width: 720
I/ACodec_vilte: setMTKParameters, height: 480
I/general: [33] Configuring video encoder - 450 frames
I/general: [33] Video encoding in progress
D/MPEG4Writer: start+ 797
I/MPEG4Writer: limits: 4294967295/0 bytes/us, bit rate: -1 bps and the estimated moov size 3191 bytes
D/MPEG4Writer: start+ 2403
D/MPEG4Writer: start- 2481
D/MPEG4Writer: start- 964
I/MPEG4Writer: setStartTimestampUs: 166665
I/MPEG4Writer: Earliest track starting time: 166665
D/MPEG4Writer: Video mStartTimestampUs=166665us
D/MPEG4Writer: reset+ 1081
D/MPEG4Writer: Video track stopping. Stop source
D/MPEG4Writer: Video track source stopping
D/MPEG4Writer: Video track source stopped
I/MPEG4Writer: Received total/0-length (439/0) buffers and encoded 439 frames. - Video
D/MPEG4Writer: Video track stopped. Stop source
D/MPEG4Writer: Stopping writer thread
D/MPEG4Writer: 0 chunks are written in the last batch
D/MPEG4Writer: Writer thread stopped
I/MPEG4Writer: Ajust the moov start time from 166665 us -> 166665 us
I/MPEG4Writer: The mp4 file will not be streamable.
D/MPEG4Writer: reset- 1187
D/MPEG4Writer: reset+ 1081
D/MPEG4Writer: Video track stopping. Stop source
I/general: [33] MP4 video created successfully

Answer 1

Why does the video have corrupted frames? From a look at the shared bytes:

1) Every frame is a Key-frame
I wondered why your MP4's section for listing keyframes, the stss atom, had every single frame listed as a type "Key-frame" (why no P-frames and B-frames?) but then I see that your code has this command:

mMediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1);

Since you are muxing (copy/pasting existing media bytes), that media has its own interval which should be written into the output MP4... A Muxing process does not need to be given a new keyframe interval.

solution:
Test a possible fix of disabling (or comment //) that line,
With same // disabling of the bitrate setting.

2) Your first frame is not a keyframe
The first video frame that you send to the muxer is not a keyframe.
It is marked as a key-frame in the MP4 metadata (at the stss atom) but it's actual bytes are of type P-frame. This means: After start-code 0, 0, 0, 1 your NAL header is found to be hex: 42, or decimal: 65). Be aware because a decimal 65 is not same as hex 65 and so it can be confusing, if you forget to convert, and think that decimal 65 means this NALU is a keyframe.

solution:
Make sure you send a NAL unit with decimals: 0, 0, 0, 1, 101 (or in hex: 00, 00, 00, 01, 65).

The 101 marks it as a keyframe NALU (header byte value is: 0x65 or as decimal value: 101).

3) You have wrong SPS and PPS
Once again the problem is a wrong SPS and PPS...
Inside the MP4, the SPS/PPS data goes into the avcC section (which holds the "AVC configuration", and AVC is just an alternate name for H264).

If I manually overwrite a part of the avcC section by copy/pasting over the bytes from the raw NALUs then your corrupt MP4 shows a perfect picture (not messy colours).

solution:

option A: Try to set a MediaFormat with the SPS and PPS included.
This means putting an entry for Codec-Specfic Data according to these keys:

Format      CSD buffer #0                       CSD buffer #1                       CSD buffer #2
H.264 AVC   SPS (Sequence Parameter Sets)       PPS (Picture Parameter Sets)        Not Used

option B: In a worst case scenario you might have to manually fix the bytes after the MP4 file is created by your Android code (eg: after the file creation code finishes, run another function which finds and overwrites some circa 25 bytes of existing SPS & PPS inside the newly created MP4). Hopefully not needed as a solution.

I cannot test Android code, please check if this example code creates a working MP4:

(1) Converting between Decimals into Hex (since bytes are written as hex)

//# check Byte as Decimal
int temp_int = (NALU_SPS[4] & 0xFF); //where NALU_SPS is an Array.
System.out.println("NALU Decimal: " + temp_int );

//# check Byte as Hex
String temp_str = Integer.toString( temp_int, 16 );
temp_str = (temp_str.length() < 2) ? ("0"+temp_str) : temp_str;
System.out.println("NALU Byte: 0x" + temp_str );

Use the above to convert your Array values when double-checking their byte value, since tutorials and H264 Specification will mention the byte values in a Hex format.

A byte example like 0xFF is hex FF and is decimal 255.

(2) Try this MediaFormat setup.

//# Set up MediaCodec
mMediaFormat = MediaFormat.createVideoFormat(codecName, width, height);
mMediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, framerate);

//# Your SPS/PPS in an Array. Can be: byte[] ...or Else: int[]
byte[] NALU_SPS = { 0x00, 0x00, 0x00, 0x01, 0x67, 0x42, 0x00, 0x29, 0xE2, 0x90, 0x16, 0x87, 0xB6, 0x06, 0xAC, 0x18, 0x04, 0x1B, 0x87, 0x89, 0x11, 0x50 };
byte[] NALU_PPS = { 0x00, 0x00, 0x00, 0x01, 0x68, 0xCE, 0x3C, 0x80 };

//# send SPS/PPS to the Muxer
mMediaFormat.setByteBuffer( "csd-0", ByteBuffer.wrap( NALU_SPS ) );
mMediaFormat.setByteBuffer( "csd-1", ByteBuffer.wrap( NALU_PPS ) );

//# This might not be needed (since Muxer could get from SPS/PPS)
mMediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, colorFormat);

//# test with these disabled
//mMediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, bitrate);
//mMediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1);