Aligning audio for smooth playing with the web audio api

Question

I am currently trying to figure how to play chunked audio with the web audio API, right off the bat everything does work.. however most transitions between chunks aren't as smooth as I want them to be, there's a very very brief moment of silence between most of them.

My current loading and playback code:

        const response = await fetch(`${this.src}`)
        const reader = response.body.getReader()

        let timestamptowaituntil = 0
        let tolog = []
        let tolog2 = []

        while (true) {
            const { done, value } = await reader.read()

            if (done) {
                console.log(tolog)
                console.log(tolog2)
                console.log(this.ctx)
                break
            } else {
                let audiodata = await this.ctx.decodeAudioData(value.buffer)
                let source = this.ctx.createBufferSource()
                source.buffer = audiodata
                source.connect(this.ctx.destination)
                source.start(timestamptowaituntil, 0, audiodata.duration)
                timestamptowaituntil +=audiodata.duration
                tolog.push(audiodata)
                tolog2.push(source)
            }
        }

How could I go about eliminating these little moments of silence (or overlap)?

Edit: So far I've tried the following

Removing some milliseconds off the waiting time.
Removing the amount of time that is in the latency properties of the AudioContext.
Making a function to get the playback length of the UInt8Array form data using its bitrate (this indeed got me a slightly different result than the .duration property of an audioBuffer, but there still is tiny gaps)

score 0 · Accepted Answer · answered Dec 10 '22 at 11:10

After trying a ton of different approaches, I finally got a thought that solved the issue in the end. My new idea was to simply play the first chunk when it arrives, and meanwhile collect as many chunks as possible, whenever a chunk is collected, its chained with the previous chunk to make one bigger chunk (this way also makes it works in firefox which requires the chunk to have a header for decoding). The playback of the first chunk is stopped 0.5-1 second before the .duration property claims it would end, this way any anomalies in detecting length are avoided. At that same time, the next chunk is played.

A few things I added to my code for this is the following:

A function to concat two chunks:

const concat = (arrayOne, arrayTwo) => {
    let mergedArray = new Uint8Array(arrayOne.length + arrayTwo.length)
    mergedArray.set([...arrayOne, ...arrayTwo])
    return mergedArray
}

Extra offset when timing:

                    source.start(timestamptowaituntil, 0, audiodata.duration - .75)

                    timestamptowaituntil += (audiodata.duration - .75 + this.ctx.currentTime)

This along with some more minor edits has brought me to a solution that makes the chunk-swap impossible to hear (every now and then it is when the cpu is overloaded and the timing slowed).

After a little change of mind: You can also instead of this simply make a function that checks whether the expiry time has passed, where you need to stop the audionode, and start the new one there without timing it upfront — April Summers, Dec 10 '22 at 11:25

Aligning audio for smooth playing with the web audio api

1 Answers1

Linked