9

I'm looking into developing an application that will require live streaming of audio. I would prefer to use some cross-platform (windows/linux/BSD) open source library written in C or C++ even though writing it using the respective OSs' Sound APIs is still an option.

I have read a bit about various sound libraries, including SFML, SDL and PortAudio. Admittedly, I have not yet researched enough about Sound in FreeBSD and Linux (how similar is it between these 2?)

The main requirements will be

  1. get audio from a chosen microphone/microphones to send over the network,
  2. send data to a selection of output devices,
  3. process the sound (filtering, cleaning up noise, multiplexing streams etc) but this can be done once I have the audio data, the library itself does NOT need to be able to do any of this.
  4. have reasonably low latency

My main concern is that these mentioned APIs seem to be mainly targeted for Games (where sound is usually loaded from disk and there is not much, if any, sound recording involved rather than streamed over the network with equal importance between recording and playback.

Does anyone have any pointers/warnings/suggestions regarding these or other sound APIs or about the advantages/disadvantages of going the long way and implementing this in the respective OSs' APIs?

NOTE: while this: "Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow " DOES make the question off-topic, when one also considers "as they tend to attract opinionated answers and spam" then I do not think this question should be closed. Someone looking for such a library as described will be hard-pressed to find anything and the answers to this question practically summarize all available options. This is thus in the "generally covers...software tools commonly used by programmers;" category of accepted answers.

msam
  • 4,259
  • 3
  • 19
  • 32

5 Answers5

7

PortAudio is an excellent choice for the application you are describing -- it runs on all those platforms, is written in C, provides low-latency, and has both callback and blocking I/O options. It definitely meets your requirements, and is not particularly aimed at games. In fact, I would say there are other apis that are better for games, whereas PortAudio is better for general purpose audio I/O such as voip-like applications like yours, audio players, pro-audio applications, audio recording, software radios, etc.

Another option you might consider is RTAudio, which I am not as familiar with. My understanding is it's a bit simpler (no blocking I/O AFAIK), and has support for more platforms, including mobile OSes, though PortAudio folks are working on that.

Your question about FreeBSD vs. Linux: Linux uses ALSA, while other unixes use OSS. Both OSS and ALSA offer compatibility layers, so ALSA has OSS compatibility and vice versa, but in my experience there are bugs in both compatibility layers. Maybe things have gotten better since I last used it, though.

Some linux desktops run PulseAudio on top of ALSA. I am not sure if this is true of FreeBSD. For some reason, ALSA is configured in exclusive mode by default on most systems. While this is easy to fix in theory, the config files are weird and most users haven't done it and never will, meaning that once PulseAudio takes over, you can't access the ALSA devices directly anymore, so you may want to have a PulseAudio driver as well, unless you want to make your users change their configuration (which you might if your application requires really low latency).

I am pretty sure PortAudio supports PulseAudio, despite what it may or may not say on the website. I will ask on the mailing list and update here.

UPDATE: someone on the mailing list thought that you can use Alsa drivers to access PulseAudio. That's (great!) news to me, but there it is.

Bjorn Roche
  • 11,279
  • 6
  • 36
  • 58
2

I consider SDL and OpenAL to be both common and widely supported. Unless these don't meet your needs I'd suggest not going to a lower level since you'll lose platform independence.

The reason they seem geared towards games is simply because that is one of the harder use cases. So if you can support games chances are you'll support anything another app will want (short of studio software).

Consider your goal of low latency. Games require a very low latency to ensure sound effects are matched well with actions on the screen. I presume this is a similar reason you want this (so your sound matches your video stream and there are no pauses in the voice channel).

BTW, your perception of games is not correct. A lot of games have user voice channels for team communication. Additionally they may include procedural sound and sound effects.

I found another one call SFML which includes recording support. I don't know much about it, but I've seen it can replace SDL.

edA-qa mort-ora-y
  • 30,295
  • 39
  • 137
  • 267
  • 1
    Here's a link to the recording part! http://stackoverflow.com/questions/3056113/recording-audio-with-openal – NotKyon Jun 22 '12 at 09:02
  • unfortunately, I hadn't reaslised that SDL does not support recording (that or i'm missing something) so that is now out of the equation. Will look into OpenAL though – msam Jun 25 '12 at 12:18
  • Yeah, sorry about that, I thought SDL did recording, but I can't find the API for it either. Perhaps one of the newer releases has it, or it was a patch. – edA-qa mort-ora-y Jun 25 '12 at 15:55
  • Adding a reference to SFML, another option. – edA-qa mort-ora-y Jun 26 '12 at 04:12
  • 1
    SDL2 supports capture according to https://wiki.libsdl.org/SDL_OpenAudioDevice . SDL_OpenAudioDevice() mentions "iscapture: non-zero to specify a device should be opened for recording, not playback". – Stéphane Gourichon Apr 27 '16 at 10:09
  • SFML is not low-latency. sf::SoundStream requires a minimum of 1500 samples @ 44100, I think. I tried it with 882 samples every 1/50th sec and it failed. OpenAL cannot process sound that quickly. – Cthutu Oct 09 '17 at 14:48
2

libsoundio is a low level C library that fits your requirements.

One caveat: it does not support OSS (FreeBSD) or sndio (OpenBSD) yet.

andrewrk
  • 30,272
  • 27
  • 92
  • 113
1

I would strongly suggest you develop it on the cross platform (Linux/Mac/Windows) Qt framework and using its own Qt libraries.In the QtMultimedia module, you can use QAudioInput to capture raw audio from a microphone.You can again use QtMultimedia for processing.

Qt framework is very much optimized for multimedia and gaming application, so you wont loose on performance.

Iharob Al Asimi
  • 52,653
  • 6
  • 59
  • 97
Ashish Gaurav
  • 303
  • 1
  • 7
  • Beyond trivial beeps none of the Qt provided sound APIs is any good. Refer to my answer here: http://stackoverflow.com/questions/5600515/qt-how-to-play-sound-witout-blocking-main-thread/5602687#5602687 – edA-qa mort-ora-y Jun 22 '12 at 08:50
  • That might be true for earlier versions but i have used Qt4.7 for 2 of my games with wide range of sound effects. – Ashish Gaurav Jun 22 '12 at 12:05
  • Unless 4.7 has an entirely new sound API I wouldn't believe it could have addressed the problems I found with it. – edA-qa mort-ora-y Jun 23 '12 at 03:43
-1

linphone contains mediastreamer2 for this exact purpose

Boris
  • 1,311
  • 13
  • 39