1

Following my other post, I am wondernig if it is possible to do some process like MFCC extraction on the decoded audio packets. The code I use decode audio and video from mpeg-2 file using ffmpeg. Process on video is done using opencv as this library permits to grab frames on by one. I need to process the corresponding audio samples in the same time.

Thanks.

Community
  • 1
  • 1
Eric
  • 2,301
  • 3
  • 23
  • 30

2 Answers2

2

I've created a C++ audio engine named "Crosstalk".

Although it's referred to as an "audio engine", It's really just a real-time C++ data (floating point) processing engine. Crosstalk allows you to create and route systems in design-time and real-time. Basically, the engine takes care of all the data routing and gives you a simple platform for creating components through which the data gets processed (E.g. your "Audio Feed" component connected in parallel with the "Video Feed" component). As long as your branches are of equal total buffer length, they will be perfectly synchronized.

It's very easy to use. Here's an example of how to configure a system to play an mp3 file (The components used here are provided with the engine):

XtSystem system;
XtMp3Decoder mp3Decoder;
XtAudioDevice audioDevice;

long md = system.addComponent(&mp3Decoder);
long ad = system.addComponent(&audioDevice);

system.connOutToIn(md,0,ad,0);
system.connOutToIn(md,1,ad,1);

mp3Decoder.loadFile("../05 Tchaikovski-Swan Lake-Scene.mp3");
mp3Decoder.play();

You can check out the API documentation and licensing details here: http://www.adaptaudio.com/Crosstalk

EDIT (01-12-2012):

Crosstalk has been replaced by an open-source project called "DSPatch". DSPatch is essentially an upgraded version of the routing engine behind Crosstalk that is no longer limited to only audio processing. DSPatch allows you to create and route almost any type of process chain imaginable, and free for personal AND proprietary use :)

Marcus Tomlinson
  • 69
  • 5
Adapt Audio
  • 36
  • 2
  • Thanks for the link. I already use mencoder (more stable then ffmpeg) for decoding streams but I'll focus on your lib. – Eric Apr 09 '12 at 11:59
-1

I downloaded your library and I'm playing with it. Did you perform some kind of performance comparison with other IPC techniques like socket/localhost, message queues, circular buffers for audio streams? I'm developing a software application that receives a multichannel UDP stream (128 channels), performs FFT on a subset of it, plays one selected channel, visualizes the spectrum of 2 channels and spectrogram of one channel. Do you think that DSPatch is fast enough to use it?downloaded your library and I'm playing with it. Did you perform some kind of performance comparison with other IPC techniques like socket/localhost, message queues, circular buffers for audio streams? I'm developing a software application that receives a multichannel UDP stream (128 channels), performs FFT on a subset of it, plays one selected channel, visualizes the spectrum of 2 channels and spectrogram of one channel. Do you think that DSPatch is fast enough to use it?

Marco
  • 1
  • 3
  • You seem to have misunderstood how Stack Overflow works. Answers are for answering the question asked in the question. Not for asking the questioner additional questions. That is what comments are for. – JensB Nov 26 '21 at 22:09
  • Sorry, but I'm not able to comment because I have less than 50 reputation. – Marco Nov 27 '21 at 08:13
  • This does not provide an answer to the question. Once you have sufficient [reputation](https://stackoverflow.com/help/whats-reputation) you will be able to [comment on any post](https://stackoverflow.com/help/privileges/comment); instead, [provide answers that don't require clarification from the asker](https://meta.stackexchange.com/questions/214173/why-do-i-need-50-reputation-to-comment-what-can-i-do-instead). - [From Review](/review/late-answers/30443375) –  Nov 28 '21 at 15:26