Synchronize video and audio (preferably without JavaScript)

Question

If I have HTML5 video and audio elements, is there a clean way to keep them in sync? They should act like a video file that contains an audio track, so advancing one track manually should bring the other one along with it. Let's say the two tracks have the same duration.

I'd like a solution that works across browsers, but I don't particularly care if it works on older browsers. It would also be nice to avoid the use of JavaScript if possible. Otherwise, a dead-simple JavaScript library would be best -- something that only asks for which tracks to synchronize and takes care of the rest.

I've looked into mediagroup... but it looks like it only works in Safari. I've looked into audioTracks... but the user has to enable the feature in Firefox.

I've looked into Popcorn.js, which is a JavaScript framework that seems designed for this task... but it looks like there hasn't been any activity in over a year. Besides that, the only demonstrations I can find are of synchronizing things like text or slides to video, not audio to video.

Is it at all possible to add the audio to your video beforehand in an external program? It would be the most reliable way of syncing the two. — Glen Despaux Jr, Aug 19 '16 at 19:48
*"without JavaScript"*, I would say this is out of the question, but you basically to listen for buffer event on both elements, and stop them both. Then, continue playing when enough data is loaded. Repeat. — Adam Azad, Aug 19 '16 at 19:50
You may be asking too much of what HTML can do on its own, then... — Glen Despaux Jr, Aug 19 '16 at 19:52
@GlenDespaux is right, HTML deals with the structure of the document not its behaviour. — n0m4d, Aug 24 '16 at 12:26
@guest271314 not a requirement, just a preference. I was hoping HTML5 would take care of it... — user1475412, Aug 29 '16 at 20:32
@user1475412 Are audio, video tracks not currently playing synchronously using `.play()`? — guest271314, Aug 29 '16 at 20:33
@guest271314 they fall out of sync: http://html5demos.com/two-videos — user1475412, Aug 29 '16 at 20:35
@guest271314 I am trying to synchronize video and audio. Here are links: goo.gl/7gI4jC and goo.gl/7gI4jC — user1475412, Aug 30 '16 at 01:16

guest271314 · Accepted Answer · 2016-08-31T01:22:30.227

You can use Promise.all(), fetch() to retrieve media resource as a Blob, URL.createObjectURL() to create a Blob URL of resource; canplaythrough event and Array.prototype.every() to check if each resource can play; call .play() on each resource when both resources can play

I didn't make it clear in the question. I meant that the two tracks should stay in sync as though playing a regular video file with audio.

One approach could be to create a timestamp variable, utilize seeked event to update .currentTime of elements when set variable is greater than a minimum delay to prevent .currentTime being called recursively.

<!DOCTYPE html>
<html>
<head>
</head>
<body>
  <button>load media</button>
  <br>
  <div id="loading" style="display:none;">loading media...</div>
  <script>
    var mediaBlobs = [];
    var mediaTypes = [];
    var mediaLoaded = [];
    var button = document.querySelector("button");
    var loading = document.getElementById("loading");
    var curr = void 0;
    var loadMedia = () => {
      loading.style.display = "block";
      button.setAttribute("disabled", "disabled");
      return Promise.all([
        // `RETURN` by smiling cynic are licensed under a Creative Commons Attribution 3.0 Unported License
        fetch("https://ia600305.us.archive.org/30/items/return_201605/return.mp3")
      , fetch("http://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4")
        ])
        .then(responses => responses.map(media => ({
          [media.headers.get("Content-Type").split("/")[0]]: media.blob()
        })))
        .then(data => {
          for (var currentMedia = 0; currentMedia < data.length; currentMedia++) {
            (function(i) {
              var type = Object.keys(data[i])[0];
              mediaTypes.push(type);
              console.log(data[i]);
              var mediaElement = document.createElement(type);
              mediaElement.id = type;
              var label = document.createElement("label");
              mediaElement.setAttribute("controls", "controls");
              mediaElement.oncanplaythrough = () => {
                mediaLoaded.push(true);
                if (mediaLoaded.length === data.length 
                   && mediaLoaded.every(Boolean)) {
                    loading.style.display = "none";
                    for (var track = 0; track < mediaTypes.length; track++) {
                      document.getElementById(mediaTypes[track]).play();
                      console.log(document.getElementById(mediaTypes[track]));
                    }
                }
              }
              var seek = (e) => {
                if (!curr || new Date().getTime() - curr > 20) {
                  document.getElementById(
                    mediaTypes.filter(id => id !== e.target.id)[0]
                  ).currentTime = e.target.currentTime;
                  curr = new Date().getTime();
                }
              }
              mediaElement.onseeked = seek;
              mediaElement.onended = () => {
                for (var track = 0; track < mediaTypes.length; track++) {
                  document.getElementById(mediaTypes[track]).pause()
                }
              }
              mediaElement.ontimeupdate = (e) => {
                e.target.previousElementSibling
                .innerHTML = `${mediaTypes[i]} currentTime: ${Math.round(e.target.currentTime)}<br>`;
              }
              data[i][type].then(blob => {
                mediaBlobs.push(URL.createObjectURL(blob));
                mediaElement.src = mediaBlobs[mediaBlobs.length - 1];
                document.body.appendChild(mediaElement);
                document.body.insertBefore(label, mediaElement);
              })
            }(currentMedia));
          }
        })
    };
    button.addEventListener("click", loadMedia);
  </script>
</body>
</html>

plnkr http://plnkr.co/edit/r51oh7KsZv8DEYhpRJlL?p=preview

I think the missing piece here is *keeping* the two tracks in sync. They should not fall out of sync and advancing one track manually should bring the other one along with it. — user1475412, Aug 30 '16 at 01:05
@user1475412 _"and advancing one track manually should bring the other one along with it."_ This description of requirement does not appear at actual Question? — guest271314, Aug 30 '16 at 01:13
Sorry I didn't make it clear in the question. I meant that the two tracks should stay in sync as though playing a regular video file with audio. I'll modify the question. — user1475412, Aug 30 '16 at 01:18

score 0 · Answer 2 · answered Aug 29 '16 at 20:14

0

To my knowledge, this is impossible with pure HTML.

You can use the currentTime property of both <video> and <audio> elements to synchronize the time.

var video = document.getElementById("your-video");
var audio = document.getElementByid("your-audio");
audio.currentTime = video.currentTime;

If necessary, you could also use the timeupdate event to continuously re-sync the video and audio.

answered Aug 29 '16 at 20:14

Shadowfacts

1,038
10
22

I think this method doesn't work very well... See http://html5demos.com/two-videos – user1475412 Aug 29 '16 at 20:16
Looking at the source of that demo, it looks like the times are only synchronized once. It might work better if the `timeupdate` event is used to continuously re-sync them. – Shadowfacts Aug 29 '16 at 20:22

Synchronize video and audio (preferably without JavaScript)

2 Answers2

Linked