HTMLMediaElement.currentTime does not appear to advance between requestMediaFrame callbacks

Question

I have a fairly simple iteration that copies the content of a <video> into a <canvas>. The idea is that I start copying when the video plays and stops copying when it detects that the video has stopped playing (whether via ending or when paused).

play() {
    this.video.play()
    this.watch()
}

watch(previous = -1) {
    requestAnimationFrame(() => {
        const current = this.video.currentTime
        this.ctx.drawImage(...)
        if(current != previous) {
            this.watch(current)
            console.dir({ current, previous })
        } else {
            console.error({ current, previous })
        }
    })
}

If there is no current != previous check, the video copies fine, but it obviously doesn't ever stop copying. If the check is in place, the output reports that the first iteration is different (previous == -1 and current == 0). But the second loop through (which is in the requestAnimationFrame callback) has previous == 0 and also current == 0, so the loop stops.

I don't understand why currentTime is not advancing. I've called play() on the video, so it should be playing, especially be the time the next animation frame comes around. I've also tried making watch() the listener for the video's play event, which didn't make a difference.

In addition to the solution in the posted answer (check for `paused` instead), you may want to use `video.requestVideoFrameCallback` where available. And globally, you may be interested in [this Q/A](https://stackoverflow.com/questions/32699721/javascript-extract-video-frames-reliably#). — Kaiido, Jan 12 '22 at 05:31

score 1 · Answer 1 · answered Jan 11 '22 at 17:11

1

Browsers don't give you precise timestamps to avoid fingerprinting, so the currentTime likely won't be frame-accurate. Also, requestAnimationFrame goes at the framerate of the page, not the video content - if you want to read by video frames there's the new WebCodecs API.

Check the video's paused or ended status instead of current position to know if it's done. You could also add event listeners for pause/ended.

Here's an example of doing what you want using async and promises:

async play() {
    // returns a promise that rejects if you have audio mute problems
    await this.video.play();
    // loop until video is no longer playing. Consider including a timeout or AbortController just in case, but not totally necessary...
    while (!(this.video.paused || this.video.ended)) {
        // wrap requestAnimationFrame in promise
        await new Promise(done => {
            requestAnimationFrame(() => {
                this.ctx.drawImage(...);
                done();
            });
        });
    }
}

answered Jan 11 '22 at 17:11

Luke

919
5
8

That's disappointing. I'm not looking for frame-accuracy on the source, I just wanted to minimize video effects when copying. I figured there would be some rounding, but I also figured there would be some noticeable amount of motion on the video progress once it was playing. Ah well. Thank you for your answer. – jyurek Jan 11 '22 at 21:21
"to avoid fingerprinting" no, that's not the reason. What would a precise timestamp give on the user? – Kaiido Jan 12 '22 at 05:29
@Kaiido the link you included in your comment above is great! By fingerprinting I was referring to [this note in MDN](https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/currentTime#reduced_time_precision) regarding the `currentTime` parameter. My point is that you can't necessarily use `currentTime` to get the current frame of the video – Luke Jan 12 '22 at 16:18
The limited timing accuracy you're talking about is much more fine-grained, and is on the order of milliseconds, not whole frames. Playing preloaded audio with the web audio API and a buffer source gives much much more accurate timestamps (at least on mobile Chrome). If this was caused by limiting timer accuracy then this would be affected too. He's just seeing a bug. – Glenn Maynard Feb 28 '23 at 02:51

score 0 · Answer 2 · answered Jan 13 '22 at 15:36

Here is a simple setup to draw a video and stop drawing when paused or ended.
See if it helpful to adjusting your own code's logic.

PS: You likely want requestAnimationFrame to happen on a Timer rather than relying on video.currentTime to update as a reason to draw new frame. Time value could be updated slower than the actual video FPS.

<!DOCTYPE html>
<html>
<body>

<video id="myVideo" width="320" height="240" controls>
<source src="movie.mp4" type="video/mp4">
</video>

<canvas id="myCanvas" width="320" height="240" style="border:1px solid #d3d3d3;">
</canvas>


<script>

var c = document.getElementById("myCanvas");
var ctx = c.getContext("2d");
const myvid = document.getElementById("myVideo");

myvid.addEventListener("play", handle_Media_Events );
myvid.addEventListener("pause", handle_Media_Events );
myvid.addEventListener("ended", handle_Media_Events );

var my_animFrameReq;

var requestAnimationFrame = (window.requestAnimationFrame || window.mozRequestAnimationFrame ||
                            window.webkitRequestAnimationFrame || window.msRequestAnimationFrame);

var cancelAnimationFrame = (window.cancelAnimationFrame || window.mozCancelAnimationFrame);


function on_timeUpdate()
{ my_animFrameReq = requestAnimationFrame(draw_video); }

function draw_video() 
{ ctx.drawImage(myvid, 0, 0, 320, 240); }

function handle_Media_Events()
{
    if ( (event.type == "pause") || (event.type == "ended") )
    { 
        cancelAnimationFrame( my_animFrameReq );
        myvid.removeEventListener("timeupdate", on_timeUpdate ); 
    }
    
    if ( event.type == "play" )
    { myvid.addEventListener("timeupdate", on_timeUpdate ); }
    
}

</script>

</body>
</html>

score 0 · Answer 3 · answered Jan 13 '22 at 20:43

As it turns out, I was asking about an XY Problem. I thought this was going to help out in the situation where seeking was not drawing the correct frame. Though in the end I think this might be slightly related, the answers I actually needed were to 1) do what @Luke said above, and play while the video was unpaused:

watch() {
  requestAnimationFrame(() => {
    this.copyFrameToCanvas()
    if(!this.video.paused) { // paused is true when ended or paused
      this.watch()
    }
  })
}

But also 2) to watch for the seeked event and, when seeking, don't update the frame until the event comes in.

seek() {
  this.video.currentTime = $time
}

this.video.addEventListener("seeked", () => this.copyFrameToCanvas())

Sorry for looking for an answer to a question I didn't ask.

HTMLMediaElement.currentTime does not appear to advance between requestMediaFrame callbacks

3 Answers3