5

There's a few related questions and discussions on this subject:

I am feeding camera preview frames (NV21 converted to NV12) to MediaCodec encoder (NV12 aka COLOR_FormatYUV420SemiPlanar). It looks like that on some devices with QualComm encoder which are running Android versions less than 4.3 I have to do some input frames processing in order to receive back frame with correct color.

On Sony Xperia ZR running Android 4.2.2 I have to add Y plane alignment in order to make it working on almost all resolutions. Code above adds 1024 bytes alignment for widths which can not be divided by 32 and 2048 bytes alignment for other resolutions. It makes MediaCodec to encode frames properly for all resolutions which can be divided by 16 (except 176x144 for which UV plane looks misaligned).

int getYPadding() {
    if (mediaCodecInfo.getName().contains("OMX.qcom") && android.os.Build.VERSION.SDK_INT < 18) {
        if ((getWidth() % 32) != 0) {
            return (getWidth()*getHeight()) % 1024;
        } else {
            return (getWidth()*getHeight()) % 2048;
        }
    }
    return 0;
}

I've tried to test this alignment on LG G2 which is running same Android 4.2.2 and has QualComm encoder, and it looks like it does not working on it correctly. UV plane is misaligned (a green stripe at the bottom of the frame). I was not able to calculate the padding which will work for both phones.

I also have access to Sony Xperia Z1 running Android 4.3 with QualComm chipset and it looks like it does not have such problems. Video on every resolution looks fine and Y plane does not needs to be aligned anyhow.

I understand that it's hardware-related and might be complicated, but since I have to support users running Android prior to 4.3 I have a question. Is it possible to programmatically determine Y plane alignment and vertical / horizontal stride values which encoder is expecting for the given color format?

Community
  • 1
  • 1
Andrey Chernih
  • 3,513
  • 2
  • 27
  • 27

1 Answers1

8

The problem in a nutshell: there were no CTS tests for video encoding until Android 4.3 (API 18).

As a result, the behavior of MediaCodec across different devices was inconsistent, and a few bugs went unnoticed. The EncodeDecodeTest tests exercise the functions you're asking about, and as a result you can reliably feed YUV data to a 4.3+ device (though you still have to runtime-detect whether it wants planar or semi-planar).

For your specific question, the Y plane on older Qualcomm devices needs to be aligned at a 2K boundary, which isn't quite what your code is doing. For 720p video this happens naturally (720*1280 == 450 * 2048), for 176x144 you'd adjust by 1280 to start the UV plane at 26624 instead of 25344. You need to set the absolute alignment within the buffer, not a fixed amount of padding -- use uvoffset = (width*height + 2047) & ~2047.

You will need to detect the codec vendor and Android software version, and if it's Qualcomm on pre-4.3 you need to make this adjustment. If your requirements change, and you can target API 18+, these issues go away. (And you can use Surface input to MediaCodec, which avoids the U/V swap issue, though depending on your needs that may not be useful.)

fadden
  • 51,356
  • 5
  • 116
  • 166
  • Thanks for the answer, @fadden You're awesome! :) With correct way of aligning I finally able to make it working for all resolutions on `Sony Xperia ZR`. It's still buggy on `LG G2`, but I think I'll wrap it up into a separate question and attach some screenshots. – Andrey Chernih Dec 20 '13 at 18:17
  • 1
    Adding on to this, more likely you would like to align Y plane and UV plane too at a 2K boundary. E.g. for a RGB-video resolution of say 480x480, the yuv420Buffer that is to be passed to MediaCodec should be of size: YUVBufferSize + yOffset + uvOffset which translates to: (480 * 480 * 1.5) + (2048 - ((480 * 480) % 2048)) + (2048 - ((480 * 480 / 2) % 2048)). I don't know which Encoder does LG_G2 use, if it's qcom-encoder, then you may try this logic. Hope this helps. – Gagan Feb 19 '14 at 19:43