Using NAudio to create AudioClip in Unity, freezes 1-2 seconds while reading data

Question

I am currently using the below code in my Unity project to stream mp3 files from a directory. It works great however, it freezes while the file is read in and the float is filled. As this project is in VR the freezes are very jarring. How can I resolve it so that it can read in without the lockup? Do I try and put it on another thread?

When commenting out the below lines there are no hitches.

aud = new AudioFileReader(musicPath);

aud.Read(AudioData, 0, (int)aud.Length);

    public void LoadSong(string musicPath){

    //Set title of song
    songTitle = Path.GetFileNameWithoutExtension(musicPath);
    if(songTitle != currentlyPlaying && songTitle != lastPlayedTitle){
        //Parse the file with NAudio
        aud = new AudioFileReader(musicPath);

        //Create an empty float to fill with song data
        AudioData = new float[aud.Length];
        //Read the file and fill the float
        aud.Read(AudioData, 0, (int)aud.Length);

        //Create a clip file the size needed to collect the sound data
        craftClip = AudioClip.Create(songTitle, (int)aud.Length, aud.WaveFormat.Channels, aud.WaveFormat.SampleRate, false);
        //Fill the file with the sound data
        craftClip.SetData(AudioData, 0);

        if(craftClip.isReadyToPlay){
            playMusic(craftClip, songTitle);
            aud.Dispose();
         }

        }

        else
        {
            playMusic(lastPlayedAudioFile, lastPlayedTitle);
        }

}

I have also tried to download the file as mentioned in this question https://answers.unity.com/questions/933090/wwwgetaudioclip-streaming-of-mp3-on-this-plattform.html using the below code and I receive the Streaming of 'mp3' on this platform is not supported error despite including the file type. Here is the code I used.

public class AudioDownloadTest : MonoBehaviour {
public AudioSource playThis;
public AudioClip clipy;
string pathh = @"D:\TestFiles\IntheAirTonightShort.mp3";

IEnumerator Download(string pathh){
    WWW song = new WWW(pathh);
    yield return song;
    clipy = song.GetAudioClip(false,false,AudioType.MPEG);
    playThis.clip = clipy;
}

void Start () {
    StartCoroutine(Download(pathh));
}

}

I have managed to improve the situation slightly by storing the last played song so that if the user selects the previous song that was played there is no delay.

Many thanks

Answer 1

Audio is usually loaded with the WWW.GetAudioClip function but since you ran into exceptions with it, you decided to use NAudio. The freezing that happens when AudioFileReader(musicPath) and aud.Read(AudioData, 0, (int)aud.Length) executes makes sense since this constructor and function are trying to load an audio file on the main thread and creating a PCM data with them.

Because AudioFileReader is not a Unity API, you should be able to use it in from Thread. Once you're doing reading the audio's float data from another Thread, you can then do a callback to the main Thread to create AudioClip with the AudioClip.Create function since you can't use this API from the main Thread.

Grab UnityThread that enables easy callback on the main thread from this post then see below for what your new LoadSong function should look like. I used ThreadPool but you can use any other Thread API in C# if you wish.

void Awake()
{
    //Enable Callback on the main Thread
    UnityThread.initUnityThread();
}

public void LoadSong(string musicPath)
{
    ThreadPool.QueueUserWorkItem(delegate
    {
        //Set title of song
        songTitle = Path.GetFileNameWithoutExtension(musicPath);
        if (songTitle != currentlyPlaying && songTitle != lastPlayedTitle)
        {
            //Parse the file with NAudio
            aud = new AudioFileReader(musicPath);

            //Create an empty float to fill with song data
            AudioData = new float[aud.Length];
            //Read the file and fill the float
            aud.Read(AudioData, 0, (int)aud.Length);

            //Now, create the clip on the main Thread and also play it
            UnityThread.executeInUpdate(() =>
            {
                //Create a clip file the size needed to collect the sound data
                craftClip = AudioClip.Create(songTitle, (int)aud.Length, aud.WaveFormat.Channels, aud.WaveFormat.SampleRate, false);
                //Fill the file with the sound data
                craftClip.SetData(AudioData, 0);

                if (craftClip.isReadyToPlay)
                {
                    playMusic(craftClip, songTitle);

                    /*Disposing on main thread may also introduce freezing so do that in a Thread too*/
                    ThreadPool.QueueUserWorkItem(delegate { aud.Dispose(); });
                }
            });
        }
        else
        {
            UnityThread.executeInUpdate(() =>
            {
                playMusic(lastPlayedAudioFile, lastPlayedTitle);
            });
        }
    });
}