Controlling MP4 Playback using javascript

Question

I am trying to clone the following site:

https://www.apple.com/mac-pro/

I am still in the prototyping phase and the big ticket item I am trying to figure out is how they are playing their MP4 file backwards when you scroll up the page. If you scroll down the page a few steps and then back up, you will see what I mean.

So far I have tried the following techniques:

Tweening currentTime property of video element
Using requestAnimationFrame and using the timestamp in the callback to update the currentTime property to the desired value

Using the requestAnimationFrame technique, I am now getting a partially usable result in every browser other than Chrome. Chrome is ok if you want to rewind maybe .5 seconds, but any more than that and it will get jumpy.

I have also made the following discoveries:

Chrome hates trying to rewind an MP4 file
As much as Chrome hates rewinding MP4 files, also make sure that you don't have an audio track on your video file. It will make it even slower.

So I feel I have a pretty good understanding of the options available to me, but the one thing that makes me think I am missing something is that the Apple website functions ok in Chrome.

I have started debugging their code which is located at:

https://images.apple.com/v/mac-pro/home/b/scripts/overview.js

And from what I can tell they seem to be using requestAnimationFrame, but I can't understand why they are getting a better result.. Does anyone have any ideas on how to achieve this effect?

BTW - I understand that videos are not really meant to be played backwards and they will never play predictably backwards. I have even had occasions on the Apple website where the rewinding can be jerky. But they still have good 2-3 second rewind transitions and the result is definitely acceptable.

Here is my relevant javascript and HTML so far..

var envyVideo, currentVideoTrigger = 0,
    currentIndicator, startTime, vid, playTimestamp, playTo, playAmount, triggeredTime, rewindInterval;

$(function() {

    vid = document.getElementById("envy-video");

    $("#play-button").click(function() {
        vid.play();
    });

    $("#rewind-button").click(function() {
        vid.pause();
        playTo = parseFloat($("#play-to-time").val());
        playAmount = playTo - vid.currentTime;
        triggeredTime = vid.currentTime;
        requestAnimationFrame(rewindToPointInTime);
    });

});

function rewindToPointInTime(timestamp) {

    if (!playTimestamp) playTimestamp = timestamp;
    var timeDifference = (timestamp - playTimestamp) / 1000;
    vid.currentTime = triggeredTime + (playAmount * (timeDifference / Math.abs(playAmount)));

    if (vid.currentTime > playTo) {
        requestAnimationFrame(rewindToPointInTime);
    } else {
        playTimestamp = null;
        playAmount = null;
    }

}



<!DOCTYPE html>
<html lang="en">           
    <head>
        <meta charset="UTF-8">
        <title>Rhino Envy</title>
        <script src="https://code.jquery.com/jquery-3.2.1.min.js" integrity="sha256-hwg4gsxgFZhOsEEamdOYGBf13FyQuiTwlAQgxVSNgt4=" crossorigin="anonymous"></script>
        <script src="./js/envy.js"></script>
        <link rel="stylesheet" href="./css/envy.css">
    </head>
    <body;
        <div id="envy-video-container">
            <video id="envy-video" src="./videos/prototype_animation.mp4"></video>
        </div>
        <div id="video-controls">   
            <p id="video-current-time"></p>
            <div class="video-control"><button id="rewind-button">rewind to</button><input type="text" id="play-to-time" placeholder="forward time" value="0"></div>
            <button id="play-button">play</button>
        </div>
        <ul id="envy-steps">
            <li id="envy-step-indicator-1"></li>
            <li id="envy-step-indicator-2"></li>
            <li id="envy-step-indicator-3"></li>
        </ul>
        <section id="envy-full-range">
            <div id="envy-1-door-link"></div>
            <div id="envy-2-door-link"></div>
            <div id="envy-3-door-link"></div>
        </section>
    </body>
</html>

score 0 · Answer 1 · answered May 09 '17 at 03:57

One solid way I can think of, would be to use two videos : one in normal direction, and the other one reversed.

You could then simply switch between which video is to be played, and only update the currentTime of the hidden one in the background.

With this solution, you can even rewind audio !

To reverse a video, you can use ffmpeg's command
ffmpeg -i input.mp4 -vf reverse -af areverse reversed.mp4

Note: You may feel some gaps at the switch, which could probably be leveraged by using a single visible video element, fed from the other's element streams, but I'll leave it for an update, I'm short on time r.n.

const vids = document.querySelectorAll('video');
vids.forEach(v => {
  v.addEventListener('loadedmetadata', canplay);
  v.addEventListener('timeupdate', timeupdate);
});
let visible = 0;

function timeupdate(evt) {
  if (this !== vids[visible]) return;
  let other = (visible + 1) % 2;
  vids[other].currentTime = this.duration - this.currentTime;
}

document.getElementById('switch').onclick = e => {
  visible = (visible + 1) % 2;
  show(vids[visible]);
}
// waith both vids have loaded a bit
let loaded = 0;
function canplay() {
  if (++loaded < vids.length) return;
  hide(vids[1]);
  show(vids[0]);
}

function show(el) {
  el.muted = false;
  const p = el.play();
  el.style.display = 'block';
  const other = vids[(visible + 1) % 2];
  // only newest chrome and FF...
  if (p && p.then) {
    p.then(_ => hide(other));
  } else {
    hide(other);
  }
}

function hide(el) {
  el.muted = true;
  el.pause();
  el.style.display = 'none';
}

document.getElementById('pause').onclick = function(e) {
  if (vids[visible].paused) {
    this.textContent = 'pause';
    vids[visible].play();
  } else {
    this.textContent = 'play';
    vids[visible].pause();
  }
}

<button id="switch">switch playing direction</button>
<button id="pause">pause</button>
<div id="vidcontainer">
  <video id="normal" src="https://dl.dropboxusercontent.com/s/m2htty4a8a9fel1/tst.mp4?dl=0" loop="true"></video>
  <video id="reversed" src="https://dl.dropboxusercontent.com/s/lz85k8tftj2j8x6/tst-reversed.mp4?dl=0" loop="true"></video>
</div>

Thanks.. I was thinking of a similar approach, but I didn't want to double up on videos, but its probably not a massive blow as videos will only be on desktops and mobile will be image based. One thing I noticed on the Apple site is that they have image placeholders at the end of each transition that is overlayed on top of the video, so this would probably ease the visual jump when moving between forward and reverse - Still it would be nice to be able to do it with a single video file.. — Nathan Wallis, May 09 '17 at 23:37
@NathanWallis and Apple's video has an uniform black background, with little movement on the video. But I guess passing an other video in their script will reveal a lot more their flows. Unfortunately, there is currently no real way in WebAPIs to play a video file in reverse. If your video are small enough, you could also try to [extract all the frames](http://stackoverflow.com/questions/32699721/javascript-extract-video-frames-reliably/32708998) before-hand and then display it on a canvas, but it would only work for small vids since the memory impact would be huge, to say the least. — Kaiido, May 10 '17 at 00:32
Good idea - I overrode the page script and inserted my own video, and it definitely tries to play it backward, however mine is more jerky. I then tried a completely different video: http://www.html5rocks.com/tutorials/video/basics/Chrome_ImF.mp4 And that rewinds nicely using the Apple code, so I will investigate the difference in codec and bit rate to see if that is what is enabling it... interesting.. oh.. and the html5rocks video has audio, so scrap my theory on audio making it chug... — Nathan Wallis, May 10 '17 at 04:56
Final update - looks like they are using the canvas which is suprising, but the only way I could see it working consistenly.. — Nathan Wallis, May 10 '17 at 05:08
Wrong again - Just got back into this project and they are just using setInterval and Date() to measure the time passed. Upon replacing their video with mine, and got a much better result (in Chrome) than I did using requestAnimationFrame... interesting... Hope that helps someone else.. — Nathan Wallis, May 15 '17 at 22:58

Controlling MP4 Playback using javascript

1 Answers1