9

I am trying to capture video at 60 FPS from a web app in Chrome Android. I have the appropriate video.frameRate constraint set for the call to getUserMedia. Chrome recognizes the frameRate setting and reports through the video track setting that it is capturing at 60 FPS but visually the video is only 30 FPS and not 60. On the same device through the stock Android Camera app it captures smooth 60 FPS video at the same resolution as I am trying through the web app.

I am testing this with Chrome Android 84 on a Pixel 3a. I have also tested on other phones at resolutions that those phones support at 60 FPS and regardless it only captures at 30 FPS.

Below is example code that demonstrates this behavior.

How can I achieve this? Or is capturing at 60 FPS not currently possible with Chrome Android?

<html>
<head>
    <title>frameRate</title>
    <script>
        if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) {
            var constraints = {
                video: {
                    width: { exact: 1920 },
                    height: { exact: 1080 },
                    facingMode: { ideal: "environment" },
                    frameRate: { exact: 60 }
                }
            };
            navigator.mediaDevices.getUserMedia(constraints).then(function (stream) {
                var video = document.getElementById('video');
                video.srcObject = stream;
                video.play();
                document.getElementById('frameRate').innerHTML = 'frameRate: ' + stream.getTracks()[0].getSettings().frameRate;
            });
        }
    </script>
</head>

<body>
    <p id="frameRate"></p>
    <video id="video" width="1920" height="1080"></video>
</body>
</html>
Ed Plese
  • 1,568
  • 10
  • 14
  • 1
    It seems the refresh rate of Chrome Android is 30 Hz. I base this on how often I can get a function to call itself. The simple function f() {f();} can call itself at 100s Hz, this is no problem. But if it calls itself using "setTimeout" or "requestAnimationFrame" I can't get below 30 Hz on laptop and three different phones. – Alexander Mathiasen Apr 19 '21 at 18:28
  • How did you determine ".. visually the video is only 30 FPS. "? – Alexander Mathiasen Apr 19 '21 at 18:29
  • @AlexanderMathiasen To me the 60 FPS videos stand out as being super smooth motion with a general lack of significant motion blur. It is difficult to describe. What I see on mobile is clearly not that, and also has a significant amount of overall jerkiness to it. – Ed Plese Jul 12 '21 at 01:42

1 Answers1

1

It is not using hardware accelerated video encoding so it is not able to capture at this rate due to the encoding being limited by available CPU resources. With some basic tests this maxed out all 4 cores of the Pixel 3a at 100%, leading to the choppy video capture.

MediaRecorder on Chrome Android in general supports only VP8 and VP9 video codecs for encoding. I tested this with the code at https://stackoverflow.com/a/64656254/2281056. As a bit of quirky behavior Chrome Android seems to incorrectly report that it supports types in the format ${type};codecs:${codec} so I removed these.

On the hardware side the Pixel 3a uses the Qualcomm Snapdragon 670. From the specifications it only supports video capture in the HEVC (h.265) format.

In summary Chrome Android only supports video capture in VP8 or VP9 but most Android hardware does not provide hardware acceleration for these formats, leading to poor quality capture.

Ed Plese
  • 1,568
  • 10
  • 14
  • So are there any workarounds? You mentioned that *most* Android hardware does not provide HW acceleration for VP8/VP9; are you aware of some phones which support those codecs for video capture? I looked at the Qualcomm site and couldn't find processors with that codec support, at least not the popular ones. Alternatively, are there some other mobile browsers which support HEVC? – Anderson Jul 12 '21 at 17:17
  • 1
    @Anderson I have not found any workarounds to it and I have not found any Android phones where this works. My research has shown similar results to yours where the Qualcomm processors only support HEVC. Native apps do have this capability so going with a native or hybrid app would address it but I am trying to avoid the native app in this case. – Ed Plese Jul 14 '21 at 03:14