HTML 5 audio .play() delay on mobile

Question

I just built a real-time app using socket.io where a "master" user can trigger sounds on receiving devices (desktop browsers, mobile browsers). That master user sees a list of sound files, and can click "Play" on a sound file.

The audio playback is instant on browsers. On mobiles however, there is a 0.5-2 seconds delay (my Nexus 4 and iPhone 5 about 1 second and iPhone 3GS 1-2 seconds).

I've tried several things to optimize the audio playback to make it faster on mobiles. Right now (at the best "phase" of its optimization I'd say), I combine all the mp3's together in one audio file (it creates .mp3, .ogg, and .mp4 files). I need ideas on how I can further fix / improve this issue. The bottleneck really seems to be in the hmtl 5 audio methods such as .play().

On the receivers I use as such:

<audio id="audioFile" preload="auto">
  <source src="/output.m4a" type="audio/mp4"/>
  <source src="/output.mp3" type="audio/mpeg"/>
  <source src="/output.ogg" type="audio/ogg"/>
  <p>Your browser does not support HTML5 audio.</p>
</audio>

In my JS:

var audioFile = document.getElementById('audioFile');

  // Little hack for mobile, as only a user generated click will enable us to play the sounds
  $('#prepareAudioBtn').on('click', function () {
    $(this).hide();
    audioFile.play();
    audioFile.pause();
    audioFile.currentTime = 0;
  });

// Master user triggered a sound sprite to play
socket.on('playAudio', function (audioClip) {
  if (audioFile.paused)
    audioFile.play();

  audioFile.currentTime = audioClip.startTime;

  // checks every 750ms to pause the clip if the endTime has been reached. 
  // There is a second of "silence" between each sound sprite so the pause is sure to happen at a correct time.
  timeListener(audioClip.endTime);
});

function timeListener(clipEndTime) {
  this.clear = function () {
    clearInterval(interval);
    interval = null;
  };

  if (interval !== null) {
    this.clear();
  }

  interval = setInterval(function () {
    if (audioFile.currentTime >= clipEndTime) {
      audioFile.pause();
      this.clear();
    }
  }, 750);
}

Also considered blob for each sound but some sounds can go for minutes so that's why I resorted to combining all sounds together for 1 big audio file (better than several audio tags on the page for each clip)

Answer 1

Instead of pausing / playing, I simply set the volume to 0 when it shouldn't be playing, and back to 1 when it should be playing. The Audio methods currentTime and volume don't slow the audio playback at all even on an iPhone 3GS.

I also added the 'loop' attribute to the audio element so it never has to be .play()'ed again.

It was fruitful to combine all mp3 sounds together because this solutions can work because of that.

Edit: audioElement.muted = true or audioElement.muted = false makes more sense.

Edit2: Can't control volume on user's behalf on iOS so I must pause() and play() the audio element as opposed to just muting and unmuting it.

Answer 2

Your setup is working well on desktop because of the preload attribute.

Unfortunately, here's Apple on the subject of preload:

Safari on iOS never preloads.

And here's MDN:

Note: This value is often ignored on mobile platforms.

The mobile platforms are making a tradeoff to save battery and data usage to only load media when it's actually interacted with by the user or programmatically played (autoplay generally doesn't work for similar reasons).

I think the best you're going to do is combining your tracks together, as you said you've done, so you don't have to pay the initial load-up "cost" as much.

Answer 3

I was having the same delay issue when testing in mobile. I found out what some HTML 5 games are using for audio since games demand very low latencies. Some are using SoundJS. I recommend you try that library out.

You can find a speed comparison between using the HTML Audio tag vs using SoundJS here:

http://www.nickfrazier.com/javascript/audio/ui/2016/08/14/js-sound-libraries.html

(test in mobile to hear the difference)

From my tests SoundJS is much faster.

In fact, it's Good enough to be used in a game, or for sound feedback in a user interface.

Answer 4

Old question but here is my solution using one of the answer above:

const el = document.createElement("audio");
el.muted = true;
el.loop = true;

const source = document.createElement("source");
source.src = lineSe;
source.type = "audio/mpeg";

el.appendChild(source);

// need to call this function after user first interaction, or safari won't do it.
function firstPlay() {
    el.play();
}

let timeout = null;

function play() {
    // In case user press the button too fast, cancel last timeout
    if (lineSeTimeout) {
        clearTimeout(timeout);
    }

    // Back to beginning
    el.currentTime = 0;

    // unmute
    el.muted = false;

    // set to mute after the audio finish. In my case 500ms later
    // onended event won't work because loop=tue
    timeout = setTimeout(() => {
        // mute audio again
        el.muted = true;
    }, 500);
}