Is it possible to encode to yuv422 with html5 VideoEncoder?

Question

I am trying to record a video using HTML5 VideoEncoder. There's a number of AVC profiles one can use (https://developer.mozilla.org/en-US/docs/Web/Media/Formats/codecs_parameter#av1). According to that page, 4:2:2 profiles are starting from 7A, like this:

         let videoEncoder = new VideoEncoder({
            output: (encodedChunk, config) => {
              // recording chunks using mp4box or mp4muxer
            },
            error: (error) => {
              console.log("onCodecError ", error);
            },
          });

          videoEncoder.configure({
            codec: "avc1.7A1032", // 4:2:2 profile
            width: 1920,
            height: 1280,
            hardwareAcceleration: "prefer-hardware",
            avc: { format: "avc" },
          });

Unfortunately this returns DOMException: Unsupported codec profile.

I tried the following script to discover any supported 7A profiles:

for (let i = 0; i < 256*256; i++) { 
    try {
            let config = {
                    codec: "avc1.7A" + i.toString(16), 
                    width: 1920,
                    height: 1280,
                    framerate: 25,
                    bitrate: 50_000_000,
                    avc: { format: "avc" },
            }; 
        let response = await VideoEncoder.isConfigSupported(config);
            if (response.supported) { console.log(config.codec); }
    } catch(e) {}
}

And found quite a few actually:

For example, 7A4032. Unfortunately, despite this profile works well, it results in YUV420 recording. Also it's nowhere to be found on a https://developer.mozilla.org/en-US/docs/Web/Media/Formats/codecs_parameter#av1 page, so I am afraid it's a kind of a glitch.

So, the question is, is there any way to record video with YUV422 profile?

UPD: even more weird that the same happens with VP09 codec. It's format is vp09.PP.LL.DD, where PP defines profile. So 00 and 02 are for 420, while 01 and 03 are for 422. And I can't create any 01 or 03 profile as well.

Opened a bug in chrome, judging by the comments it may be just a flaw in webcodecs implementation https://bugs.chromium.org/p/chromium/issues/detail?id=1466392 — Rustam, Jul 25 '23 at 09:01

score 1 · Accepted Answer · answered Jul 27 '23 at 16:12

So, the question is:
"Is there any way to record video with YUV422 profile?"

No, it's not possible to encode YUV422 with the WebCodecs API (at present).

422 is a packed YUV format (eg: YUY2).
420 is a planar YUV format (eg: NV12).

WebCodecs seems to process frames as 8-bit Planar YUV system (eg: I420 instead of NV12).

You cannot change it because it seems to be hardcoded. See Chromium source code line 172:

//# static
bool VideoFrame::IsSupportedPlanarFormat(media::VideoFrame* frame) 
{
    // For now only I420 in CPU memory is supported.
    return  frame && frame->IsMappable() &&
            frame->format() == media::PIXEL_FORMAT_I420 &&
            frame->layout().num_planes() == 3;
}

Look at the expected input sources:
HTMLVideoElement, HTMLCanvasElement, ImageBitmap, OffscreenCanvas, HTMLImageElement, SVGImageElement and VideoFrame.

Are those above-listed items not giving back an RGB planar output?

Also VideoFrame is your nearest option to putting custom YUY2 data but it will auto encode to 420.

The <video> tag itself in Chrome can decode YUV422 and YUV444 as tested in MP4 files. I wonder why their H264 codebase can decode them but then also cannot encode pixels as YUV422 and YUV444.

"There's a number of AVC profiles one can use..."

That shown page link seems to be talking about the general capabilities of the <video> tag.

I don't think here it considers (or reflects) the specific features of the WebCodecs API itself.

PS: I tested avc1.7A4032 using VideoEncoder.isConfigSupported and got false on Windows Chrome.

Is it possible to encode to yuv422 with html5 VideoEncoder?

1 Answers1