1

I'm using Icecast to stream live audio from internal microphones and want the listener to have as small a latency as possible.

A naive solution would be to simply access http://myhostname:8000/my_mountpoint to get the stream but the <audio> tag does internal buffering before playing and results in a pretty high latency.

Current solution: I used the ReadableStreams API to decode (using decodeAudioData of Web Audio API) and play chunks of data by routing the decoded data to an Audio Context destination (internal speakers). This works and brings down the latency significantly.

Problem: This streams API, while experimental, should technically work on the latest Chrome, Safari, Opera, FF (after setting a particular flag). I'm however having problems with decodeAudioData in all other browsers except Chrome and Opera. My belief is that FF and Safari cannot decode partial MP3 data because I usually hear a short activation of the speakers when I start streaming. On Safari, the callback on a successful decodeAudioData is never called and FF simply says EncodingError: The given encoding is not supported.

Are there any workarounds if I want to at least get it to work on Safari and FF? Is the decodeAudioData implementation actually different on Chrome and Safari such that one works on partial MP3 and the other doesn't?

Brad
  • 159,648
  • 54
  • 349
  • 530
Jon Gan
  • 867
  • 1
  • 11
  • 22

3 Answers3

2

Don't use Icecast if you need sub-second latency!

Don't use Icecast if you need sub-10-second latency and don't have full control over the whole chain of software and network!

Yes, this is an answer, not a comment. Icecast is not designed for such use cases. It was designed for 1-to-n bulk broadcast of data over HTTP in non-synchronized fashion.

What you are explaining sounds like you really should consider something that was designed to be low latency, like web-RTC.

If you think you really should use Icecast, please explain why. Because your question speaks otherwise. I'm all for more Icecast use, I'm it's maintainer after all, but its application should make sense.

TBR
  • 2,790
  • 1
  • 12
  • 22
  • I'm building a 1-to-n (where n is around 20) system that is similar to a simultaneous translation system (think one guy speaking into a sending device and visitors listening on a receiver device, except with WiFi and not radio). I've actually managed to get sub-second (around half a second, totally usable for me) latency if I used VLC as a (receiving) client and tuned it to buffer as little as possible. So I believe the problems lies more on the client side than Icecast. – Jon Gan Jan 12 '19 at 07:46
  • You *can* get fairly low latency, but you *must* control things very tightly. It is correct that Icecast doesn't add much overhead, especially if you turn off burst on connect. It's still not something that HTTP/TCP streaming is designed to be good at, as also implemented in Icecast. An Audio conferencing solution based on webRTC is probably much better at this. – TBR Jan 13 '19 at 09:48
  • Hi @TBR, Thanks for your two cent's worth. I was wondering where to start with WebRTC. I found this One-way broadcasting example: https://www.webrtc-experiment.com/webrtc-broadcasting/#7434455970902303 .. The issue is however, that I can't use a browser since I'm dealing with a USB audio interface with multiple input channels and I don't think Chromium browsers (or any browser for that matter) support multichannel audio input yet. Would WebRTC still work without a browser? – Jon Gan Jan 17 '19 at 09:01
0

I'm using Icecast to stream live audio from internal microphones and want the listener to have as small a latency as possible.

There are a few issues with your setup that prevent the lowest latency possible:

  • HTTP Progressive (the type of streaming used by Icecast and similar) runs over TCP, a reliable transport. If a packet is lost, there is some lag time while it is re-transmitted to the client, before data is delivered to the browser. This ensures every bit of audio is heard, in-order, but can cause a delay. For low latency, UDP packets are commonly used so that any lost packets can just be skipped over rather than waited for, causing a glitch, but the client keeps the latency down.

  • MP3 isn't a great codec for low latency. There are better codecs, such as Opus, which are more efficient and can generate smaller segments.

  • As you have seen, clients will buffer more by default, when streaming over HTTP.

However, in ideal conditions, I've streamed over HTTP Progressive with less than 250ms latency in Chrome, with a custom server. I bet you could get similar performance out of Icecast if you turned off the server-side buffering and used a different codec.

I'm however having problems with decodeAudioData in all other browsers except Chrome and Opera. My belief is that FF and Safari cannot decode partial MP3 data

Yes, decodeAudioData() is meant for decoding an entire file. You'll have a hard time decoding arbitrarily segmented chunks in a sample-accurate way.

Fortunately, there is a method to do what you want... MediaSource Extensions.

https://developer.mozilla.org/en-US/docs/Web/API/Media_Source_Extensions_API

Basically, you can use that exact same ReadableStream interface and push the data to a SourceBuffer to be eventually played out a normal <audio> tag. This does work with normal MP3 audio from Icecast and similar servers, provided you've worked around any cross-origin issues.

Brad
  • 159,648
  • 54
  • 349
  • 530
  • I’m quite sure that server side buffering is not the problem as I’ve used VLC too as a client (instead of web) and realized with the right settings I could bring down the end to end latency to a sub-second level. I’ve even narrowed down the issue to a x-second buffer that the HTML audio tag performs when starting the stream and that results in that x-second delay. Can the use of MediaSource actually solve that issue? – Jon Gan Mar 24 '19 at 11:18
  • @mercurial What codec are you using? I've had ~250ms latency even with just the regular ` – Brad Mar 24 '19 at 16:27
  • I tried mp3, OPUS and AAC at 320 kbps and they were all around 2-4 seconds on a local network. – Jon Gan Apr 02 '19 at 05:23
0

As @Brad mentioned, Opus is excellent for low-latency. I wrote a low-latency WASM Opus decoder that can be used with Streams, and I started another demo that demonstrates how to playback partial files with decodeAudioData():

anthumchris
  • 8,245
  • 2
  • 28
  • 53