9

I have programmed a server that creates an audio stream from my MacBook's audio input, using express, osx-audio and lame:

const http = require("http");
const express = require("express");
const audio = require("osx-audio");
const lame = require("lame");
const audioInput = new audio.Input();

const encoder = new lame.Encoder({
  channels: 2,
  bitDepth: 16,
  sampleRate: 44100,
  bitRate: 128,
  outSampleRate: 22050,
  mode: lame.STEREO
});

audioInput.pipe(encoder);

const app = express();
const server = http.Server(app);

app.get("/stream.mp3", (req, res) => {
  res.set({
    "Content-Type": "audio/mpeg",
    "Transfer-Encoding": "chunked"
  });
  encoder.pipe(res);
});

server.listen(3000);

On the client side, the sound from this audio stream is included as an <audio> element like so:

<audio controls autoplay preload="none">
  <source src="./stream.mp3" type="audio/mpeg" />
  <p>Oops – your browser doesn't support HTML5 audio!</p>
</audio>

This works – I can hear the sound from the input source I've selected on my laptop from any browser that is connected to the server when I click the “play” button on the audio element.

However the audio played by the browser lags several seconds behind the original signal. It seems although I'm using preload="none", the browser buffers quite a lot of the audio stream before it starts playing it.

Is there something obvious missing here? Is there a better way to achieve real-time audio with only a few milliseconds latency instead of several seconds?

If you're interested, my project's complete source code is here on GitHub.

Patrick Hund
  • 19,163
  • 11
  • 66
  • 95
  • I had similar issue when streaming audio with 206 responses, then I found a good solution using [MediaSource](https://developer.mozilla.org/en-US/docs/Web/API/MediaSource) which is [in this example](https://github.com/nickdesaulniers/netfix/blob/gh-pages/demo/bufferWhenNeeded.html) created for video but can easily be transformed to work audio-only. Maybe this is something, that brings you any further. – Jankapunkt May 15 '19 at 14:22
  • Thanks, I'll check it out – Patrick Hund May 17 '19 at 10:39
  • 1
    Try with another audio codec, like opus, just to see if it's a codec problem (but sincerely I don't think so). If it does not work for your purposes, you are better off using WebSockets and AudioContext API on client side. You can't control browser buffering. It's handled by the browser itself. – FonzTech May 19 '19 at 22:42

1 Answers1

4

Rather than reinventing the wheel, you can use FFMPEG, advertised as “A complete, cross-platform solution to record, convert and stream audio and video.”

Example:

ffmpeg -re -i input -f rtsp -muxdelay 0.1 rtsp://server/live.sdp

You can choose which library (h.264, mpeg, etc) your browser is OK with.

Patrick Hund
  • 19,163
  • 11
  • 66
  • 95
Dr Yuan Shenghai
  • 1,849
  • 1
  • 6
  • 19
  • Thanks, that seems like a good solution. I've taken the liberty of editing your answer a bit and awarded you the bounty. – Patrick Hund May 21 '19 at 11:16
  • Here is a short sample of my code. the code word"-preset ultrafast -tune zerolatency" is imporant also. lxterminal -e ffmpeg -f v4l2 -framerate 30 -video_size 800x448 -i /dev/video0 -i /home/pi/Desktop/sound/ic_ch.png -codec:v h264 -r 30 -s 800x448 -bf 0 -g 30 -bufsize 8000k -maxrate 8000k -filter_complex "[0:v][1:v] overlay=(W-w)/2:(H-h)/2:enable='gte(t,1)'" -preset ultrafast -tune zerolatency -f h264 udp://192.168.5.10:23003 & sleep 0.1 – Dr Yuan Shenghai May 22 '19 at 01:19
  • I finally got around to trying out your proposed solution, set up FFmpeg, streaming to an Icecast server. Alas, this does nothing at all to reduce the latency, it is even worse, so I had to un-accept the answer so as to not mislead anyone else. – Patrick Hund Aug 27 '19 at 14:59
  • not sure how you tested and whats your hardware. we tested with RPI3 and Logitec C930 for network streaming, the -preset ultrafast -tune zerolatency works well in removing the latency for our test case. If you really need proof, I can upload a video showing this later – Dr Yuan Shenghai Aug 28 '19 at 07:10
  • Those preset and tune flags are for video, not audio, though, aren't they? – Patrick Hund Aug 28 '19 at 10:23
  • This post says it's basically a problem with the browser and it's audio HTML elements - browsers will always buffer, producing latency, by design https://stackoverflow.com/a/17151097 – Patrick Hund Aug 28 '19 at 10:27
  • 1
    HTML wise I`m not very sure. I do use FFmpeg to remote send robot video and sound to the ground station to remove the latency. Yes, in ur case, it is a different issue – Dr Yuan Shenghai Aug 28 '19 at 10:59