createMediaElementSource plays but getByteFrequencyData returns all 0's

Question

I am attempting to visualize audio coming out of an element on a webpage. The source for that element is a WebRTC stream connecting to an Asterisk call via sip.js. The audio works as intended.

However, when I attempt to get the frequency data using web audio api, it returns an array of all 0's, even though the audio is working. This seems be a problem with createMediaElementSource. If I call getUserMedia and use createMediaStreamSource to connect my microphone to the input, I indeed get the frequency data returned.

This was attempted in both Chrome 40.0 and Firefox 31.4. In my search I found similar errors with Android Chrome but my versions of desktop Chrome and Firefox seem like they should be functioning correctly. So far I have a feeling that the error may be due to the audio player getting it's audio from another AudioContext in sip.js, or something having to do with CORS. All of the demos that I have tried work correctly, but only use createMediaStreamSource to get mic audio, or use createMediaElementSource to play a file (rather than streaming to an element).

My Code:

var context = new (window.AudioContext || window.webkitAudioContext)();

    var analyser = context.createAnalyser();
    analyser.fftSize = 64;
    analyser.minDecibels = -90;
    analyser.maxDecibels = -10;
    analyser.smoothingTimeConstant = 0.85;

    var frequencyData = new Uint8Array(analyser.frequencyBinCount);
    var visualisation = $("#visualisation");
    var barSpacingPercent = 100 / analyser.frequencyBinCount;
    for (var i = 0; i < analyser.frequencyBinCount; i++) {
            $("<div/>").css("left", i * barSpacingPercent + "%").appendTo(visualisation);
    }
    var bars = $("#visualisation > div");
    function update() {
            window.requestAnimationFrame(update);
            analyser.getByteFrequencyData(frequencyData);
            bars.each(function (index, bar) {
                    bar.style.height = frequencyData[index] + 'px';
                    console.debug(frequencyData[index]);
            });
    };

$("audio").bind('canplay', function() {
    source = context.createMediaElementSource(this);
    source.connect(analyser);
    update();
});

Any help is greatly appreciated.

score 4 · Accepted Answer · edited May 23 '17 at 11:59

4

Chrome doesn't support WebAudio processing of RTCPeerConnection output streams (remote streams); see this question. Their bug is here. Edit: they now support this in Chrome 50

See the test code for firefox about to land as part of this bug: Bug 1081819. This bug will add webaudio input to RTCPeerConnections in Firefox; we've had working WebAudio processing of output MediaStreams for some time. The test code there tests both sides; note it depends a lot on the test framework, so just use it as a guide on hooking to webaudio.

edited May 23 '17 at 11:59

Community

1
1

answered Mar 03 '15 at 20:25

jesup

6,765
27
32

Chrome 50 was released and should now support this. Also see: http://stackoverflow.com/questions/36581380/audio-level-meter-for-web-rtc-stream/36620646#36620646 – sqwk Apr 14 '16 at 10:44

createMediaElementSource plays but getByteFrequencyData returns all 0's

1 Answers1