Streaming large, looping audio files with Web Audio API

Question

I'm making an online game using pure Javascript, and I've encountered an issue with audio. Up until now, I've been using XMLHttpRequest and the Web Audio API to load audio files, decode them into a buffer using decodeAudioData when they are fully loaded, and store the buffer for later use. (Players are able to upload music files for their worlds, so loading everything at the beginning of the game is not an option). This solution is outlined in this topic.

This works fine for short sound files, but for large music files, there is an unacceptable loading time before the music can actually be played. I need a solution that allows streaming.

I've searched around, but generally what I see is people saying to use an Audio element and Web Audio's CreateMediaNode. This isn't optimal though, since the Audio element is imprecise and has an ugly gap when looping.

I can find plenty of examples of using createMediaStreamSource to capture audio data from a microphone, but not a single example of streaming data a file on the server, outside of playing a regular HTML media node and using it as input - which is no help, because the whole point is to start playing the music quickly.

I have been able to retrieve the audio file data over time by listening for readystatechange events from the XMLHttpRequest, but I can't figure out how to insert the new data into the buffer, or to stream the data to a Web Audio output. I can live with occasional choppiness, but I don't want any issue that will be a consistent problem.

                    this.returnobj = new BSoundSource({src:media_url});
                    this.request = createCORSRequest('GET',media_url);
                    var request = this.request;
                    request.seenBytes = 0;
                    
                    this.bindListener(loader,this.request,'readystatechange',function(){
                        var readyState = request.readyState;
                        if (readyState == 3){ //Loading
                            if (!_this.ok){
                                var newData = request.response.substr(request.seenBytes);
                                console.log("newData: <<" +newData+ ">>");
                                request.seenBytes = request.responseText.length;
                                console.log("seenBytes: " +request.seenBytes);
                                
                                _this.returnobj.updateAudioData(request); //I don't know what to do here
                            }
                        }
                    });
                    this.bindListener(loader,this.request,'load',function(e){
                        var request = e.target;
                        _this.returnobj.onAudioDataLoaded(request.response); //Uses decodeAudioData, this works fine
                        _this.done = true;
                        _this.ok = true;
                        _this.onResourceLoaded(false);
                    });

Audio element is what you want. You can fix the loop problem with MediaSource Extensions by continuously streaming data in. Also, consider using a Service Worker for caching, since you'll be using that audio source data repeatedly. — Brad, Dec 07 '20 at 05:43

anthumchris · Answer 1 · 2020-12-09T12:48:16.060

1

I've done a lot of experimentation with this, and the WebAudio API with Opus example is very low-latency. It's a POC, but you could implement similarly and it's cross-browser. Just mind the GitHub issue regarding the need to throttle the decoding:

https://fetch-stream-audio.anthum.com/ cross-browser

AudioWorklets are another example of playing (once the API comes to maturity, still a few bugs out there, but Firefox works excellently)

https://opus-bitrates.anthum.com/ not yet cross-browser

edited Dec 09 '20 at 12:48

answered Dec 08 '20 at 03:35

anthumchris

8,245
2
28
53

Could work, but not ideal since it can't handle MP3s. Even so, could you give an example showing how it could be used? – IndigoFenix Dec 09 '20 at 10:07
Any specific reason you're choosing to use MP3 instead of Opus? – anthumchris Dec 09 '20 at 11:38
Users will be uploading their own music files, so having as many options available as possible is important. – IndigoFenix Dec 09 '20 at 12:06
https://github.com/GersonRosales/Record-Audios-and-Videos-with-getUserMedia/blob/master/Mp3LameEncoder.js is mp3 / streams encoding . i link it in event that opus recording/ encode streaming helps solve and you need to add stream imple for mp3 . – Robert Rowntree Dec 09 '20 at 14:28

Streaming large, looping audio files with Web Audio API

1 Answers1