4

I am trying to buffer MP3 songs using node js and socket io in real time. I basically divide the MP3 into segments of bytes and send it over to the client side where the web audio API will receive it, decode it and start to play it. The issue here is that the sound does not play continuously, there is a something like a 0.5 seconds gap between every buffered segment. How can I solve this problem

// buffer is a 2 seconds decoded audio ready to be played 
// the function is called  when a new buffer is recieved
function stream(buffer)
{
    // creates a new buffer source and connects it to the Audio context
    var source = context.createBufferSource();
    source.buffer = buffer;
    source.connect(context.destination);
    source.loop = false;
    // sets it and updates the time 
    source.start(time + context.currentTime);

    time += buffer.duration; // time is global variable initially set to zero
}

The part where stream is called 

    // where stream bufferQ is an array of decoded MP3 data 
    // so the stream function is called after every 3 segments that are recieved 
     // the Web audio Api plays it with gaps between the sound
    if(bufferQ.length == 3)
    {
        for(var i = 0, n = bufferQ.length ; i < n; i++)
        {
              stream(bufferQ.splice(0,1)[0]);
        }
    }

should I use a different API other than the web audio API or is there a way to schedule my buffer so that it would be played continuously ?

Abdullah Emad
  • 55
  • 7
  • When is `stream()` called? – guest271314 Jun 04 '17 at 23:50
  • @guest271314 sorry I didn't clarify this point. I already edited my answer. Can you please take a look at it ? – Abdullah Emad Jun 05 '17 at 00:01
  • You could start the next `bufferSource` at the conclusion of the previous `bufferSource`. You can also utilize `MediaSource` and `updateend` event to append an `ArrayBuffer` representation of a media source segment to media playback, see [HTML5 audio streaming: precisely measure latency?](https://stackoverflow.com/q/38768375/) – guest271314 Jun 05 '17 at 00:05
  • @guest271314 I have tried to start the next bufferSource at the conclusion of the previous one by doing something like source.start(time + context.currentTime - 0.5); however, it did not work as expected. I will try to make use of the updateend. Should I create a media source instead and append new decoded buffer to it every time a segment is received ? Thanks for your help and time – Abdullah Emad Jun 05 '17 at 00:18
  • You can append an `ArrayBuffer` to a `MediaSource` `SourceBuffer` using `.appendBuffer()` at `updateend` event of [`SourceBuffer`](https://w3c.github.io/media-source/#idl-def-sourcebuffer). `decodeAudioData()` is not necessary. See https://w3c.github.io/media-source/#mediasource – guest271314 Jun 05 '17 at 00:22
  • 1
    Thanks a lot! I will read this as well as the previous stack overflow question and try it out! – Abdullah Emad Jun 05 '17 at 00:48
  • Just a question. Wouldn't a normal file server and an audio tag do the job as well? Why do you need to slice the mp3 anyway? If you use the file server and the audio tag the tag will buffer anything for you. Just thinking about it. – Kilian Hertel Jun 19 '17 at 13:36
  • of course it would ! but it wouldn't start right away unless the mp3 audio is fully loaded.. beside I am making something like an online radio, which means the audio on the client side has to be synchronized with the audio on the server side.In addition, I don't know the particulur size of the whole audio being played. That's why I need to send the audio as slices and make them as small as possible for fast playback – Abdullah Emad Jun 20 '17 at 16:46

2 Answers2

2

context.currentTime will vary depending on when it is evaluated, and every read has an implicit inaccuracy due to being rounded to the nearest 2ms or so (see Firefox BaseAudioContext/currentTime#Reduced time precision). Consider:

function stream(buffer)
{
...
source.start(time + context.currentTime);

time += buffer.duration; // time is global variable initially set to zero

Calling source.start(time + context.currentTime) for every block of PCM data will always start the playback of that block at whatever the currentTime is now (which is not necessarily related to the playback time) rounded to 2ms, plus the time offset.

For playing back-to-back PCM chunks, read currentTime once at the beginning of the stream, then add each duration to it after scheduling the playback. For example, PCMPlayer does:

PCMPlayer.prototype.flush = function() {
...
    if (this.startTime < this.audioCtx.currentTime) {
        this.startTime = this.audioCtx.currentTime;
    }
...
    bufferSource.start(this.startTime);
    this.startTime += audioBuffer.duration;
};

Note startTime is only reset when it represents a time in the past - for continuous buffered streaming it is not reset as it will be a value some time in the future. In each call to flush, startTime is used to schedule playback, and is only increased by each PCM data duration, it does not depend on currentTime.

Another potential issue is that the sample rate of the PCM buffer that you are decoding may not match the sample rate of the AudioContext. In this case, the browser resamples each PCM buffer separately, resulting in discontinuities at the boundaries of the chunks. See Clicking sounds in Stream played with Web Audio Api.

bain
  • 1,710
  • 14
  • 15
0

It's an issue with mp3 files, each mp3 file has a few frames of silence at the start and end. If you use wav files or time the start and stop of each file properly you can fix it

user7951676
  • 377
  • 2
  • 10