How to calculate FFT using NAudio in realtime (ASIO out)

Question

I am programming clone of guitar (violin) Hero as a final project for this school year.

The idea is to take input from my electric violin, analyse it via FFT, do some logic and drawing and output it through speakers. Perhaps some steps in parallel threads.

I already have Asio low latency input-output implemented but I am having a great problem implementing realtime FFT.

This is a code that sets up asioOut along with sampleAggregator. Sample aggregator should store samples that are added each time AudioAvailable() is called and trigger FFT calculation when the number of samples exceeds fftLength.

private static int fftLength = 8192;
private SampleAggregator sampleAggregator = new SampleAggregator(fftLength);

void asioStartPlaying(object sender, EventArgs e)
{
    sampleAggregator.PerformFFT = true;
    sampleAggregator.FftCalculated += new EventHandler<FftEventArgs>(FftCalculated);
    var asioOut = new AsioOut();
    BufferedWaveProvider wavprov = new BufferedWaveProvider(new WaveFormat(48000, 1));
    asioOut.AudioAvailable += new EventHandler<AsioAudioAvailableEventArgs> (asio_DataAvailable);
    asioOut.InitRecordAndPlayback(wavprov, 1, 25);
    asioOut.Play();
}

void asio_DataAvailable(object sender, AsioAudioAvailableEventArgs e)
{
    byte[] buf = new byte[e.SamplesPerBuffer*4];

    for (int i = 0; i < e.InputBuffers.Length; i++)
    {
        Marshal.Copy(e.InputBuffers[i], buf, 0, e.SamplesPerBuffer*4);
        Marshal.Copy(buf, 0, e.OutputBuffers[i], e.SamplesPerBuffer*4);
    }

    for (int i = 0; i < buf.Length; i=i+4)
    {
        float sample32 = BitConverter.ToSingle(buf, i);
        sampleAggregator.Add(sample32);
    }

    e.WrittenToOutputBuffers = true;
}

SampleAggregator is class taken from NAudio fft result gives intensity on all frequencies C#.

Asio outputs data in Int32LSB sample type. In buf there are values from 0 to 255.

This is function that should be called when fft is calculated (triggered from SampleAggregator class).

void FftCalculated(object sender, FftEventArgs e)
{
    for (var i = 0; i < e.Result.Length; i++)
    {
        Debug.WriteLine("FFT output.");
        Debug.WriteLine(e.Result[i].X);
        Debug.WriteLine(e.Result[i].Y);
    }
}

But the FFT always outputs NaN as a result.

I think there is a problem with the conversion to float.

Could someone point me in the right direction?

EDIT_1: I changed the loop in DataAvailable() to

for (int i = 0; i < e.SamplesPerBuffer * 4; i++)
{
    float sample32 = Convert.ToSingle(buf[i]);
    sampleAggregator.Add(sample32);
}

And FFT now outputs data. But I think they are not correct. The mistake must be in the conversion between asio samples and float values. But I am not much comfortable around byte operations.

Could e.GetAsInterleavedSamples somehow help?

Sample of raw data from FFT: X: -5,304741 Y: -0,7160959 X: 6,270798 Y: -0,4169312 X: -8,851931 Y: -0,4485725

I noticed, that first few and last few values in raw data from FFT are somehow bigger then other data. Making calculation of magnitude tricky.

I don't know NAudio, but why do you set `sampleAggregator.PerformFFT` to `false`? I would have guessed it has to be true to calculate the FFTs for each window. — Dirk, Jun 03 '14 at 13:34
That's a good point. I was experimenting with it and forgot to set it true in example on stackoverflow. Thanks. — Lucause, Jun 03 '14 at 13:37
Are you sure your ASIO driver is providing floating point samples? It's more common to get 16 or 24 bit integer samples. — Mark Heath, Jun 03 '14 at 16:27
I am not sure. I have found out from AsioAudioAvailableEventArgs that my AsioSampleType is Int32LSB. In buf there are values from 0 to 255 (bytes?). I just tried a change more described under EDIT_1. But I do not think that is a correct approach. — Lucause, Jun 03 '14 at 16:55

score 4 · Accepted Answer · edited Feb 28 '22 at 20:35

The problem was as I thought in conversion between data about samples from Asio (4 bytes in a row in buf array) to float for fft. BitConvertor should do the trick but it somehow makes fft output NaN in my case. So I tried this conversion instead.

for (int i = 0; i < e.SamplesPerBuffer * 4; i += 4)
{
    float sample = Convert.ToSingle(buf[i] + buf[i + 1] + buf[i + 2] + buf[i + 3]);
    sampleAggregator.Add(sample);
}

And it works very well. Even with 192 000 sample rate.

score 2 · Answer 2 · answered Jun 03 '14 at 17:39

I haven't used NAudio, but we have implemented pretty much similar thing with DirectSound. There's tools for this in LightningChart Ultimate SDK. AudioInput component captures waveform data from sound device, and the data is forwarded to FFT calculation (SpectrumCalculator component) and waveform monitors at same time. FFT data is then visualized as spectrograms in 2D or 3D. AudioOutput writes the data to sound device to be audible through speakers.

Overall the audio input/output, FFT calculation and visualization run with very low CPU load.

Spectrogram example in LightningChart demo application

Our libraries are commercial, but I think even if you were not looking for any additional components, it might be a good idea to take a look at our audio examples, source code is visible in the demo application Visual Studio projects. You may get fresh ideas, at least :-) You can apply some methods for NAudio, I believe.

Download LightningChart demo from LightningChart web site, running it costs you nothing.

[I'm CTO of LightningChart components]

Thank you for the link. Your visuals are indeed nice. But my problem is more Asio specific I believe. — Lucause, Jun 03 '14 at 17:54

How to calculate FFT using NAudio in realtime (ASIO out)

2 Answers2

Linked