Read file while updating progress bar with coroutines in Unity

Question

I'm trying to read a file line by line while I update a progress bar (two GUI textures with a float expansion in one of its width (maxWidth * currentPercentage)).

I have two implementations:

    public static string ThreadedFileRead(string path, Action<float> percAction)
    {
        FileInfo fileInfo = new FileInfo(path);
        StringBuilder sb = new StringBuilder();

        float length = fileInfo.Length;
        int currentLength = 0;

        using (StreamReader sr = new StreamReader(path))
        {
            while (!sr.EndOfStream)
            {
                string str = sr.ReadLine();
                sb.AppendLine(str);

                // yield return str;

                percAction(currentLength / length);
                currentLength += str.Length;
                Interlocked.Add(ref currentLength, str.Length);
            }

            percAction(1f);

            return sb.ToString();
        }
    }

Using the following implementation:

  // Inside a MonoBehaviour

  public void Start() 
  {
       string fileContents = "";
       StartCoroutine(LoadFileAsync(Application.dataPath + "/Data/file.txt", (s) => fileContents = s));
  }

  public IEnumerator LoadFileAsync(string path, Action<string> fin) 
  {
        string contents = "";

        lock (contents)
        {
            var task = Task.Factory.StartNew(() =>
            {
                contents = F.ThreadedFileRead(path, (f) => currentLoadProgress = f);
            });

            while (!task.IsCompleted)
                yield return new WaitForEndOfFrame();

            fin?.Invoke(contents);
        }
  }

But this blocks the current GUI (I don't know why).

I also used this:

    // Thanks to: https://stackoverflow.com/questions/41296957/wait-while-file-load-in-unity
    // Thanks to: https://stackoverflow.com/a/34378847/3286975
    [MustBeReviewed]
    public static IEnumerator LoadFileAsync(string pathOrUrl, Action<float> updatePerc, Action<string> finishedReading)
    {
        FileInfo fileInfo = new FileInfo(pathOrUrl);

        float length = fileInfo.Length;

        // Application.isEditor && ??? // Must review
        if (Path.IsPathRooted(pathOrUrl))
            pathOrUrl = "file:///" + pathOrUrl;

        /*

        using (var www = new UnityWebRequest(pathOrUrl))
        {
            www.downloadHandler = new DownloadHandlerBuffer();

            CityBenchmarkData.StartBenchmark(CityBenchmark.SendWebRequest);

            yield return www.SendWebRequest();

            CityBenchmarkData.StopBenchmark(CityBenchmark.SendWebRequest);

            while (!www.isDone)
            {
                // www.downloadProgress
                updatePerc?.Invoke(www.downloadedBytes / length); // currentLength / length
                yield return new WaitForEndOfFrame();
            }

            finishedReading?.Invoke(www.downloadHandler.text);
        }

         */

        using (var www = new WWW(pathOrUrl))
        {
            while (!www.isDone)
            {
                // www.downloadProgress
                updatePerc?.Invoke(www.bytesDownloaded / length); // currentLength / length
                yield return new WaitForEndOfFrame();
            }

            finishedReading?.Invoke(www.text);
        }
    }

With the following implementation:

  public IEnumerator LoadFileAsync(string path, Action<string> fin) 
  {
       yield return F.LoadFileAsync(path, (f) => currentLoadProgress = f, fin);
  }

The last code I shared has two parts:

• The commented part blocks also the main thread.
• The WWW class I used (it will be deprecated in a future) doesn't block the main thread, but it only displays two steps on the progress bar (like 25% and 70%).

I don't why this is happening and if there is a better approach for this.

So, any help (guidance) for this is welcome.

Answer 1

The commented part blocks also the main thread.

It is not "blocking" the main thread. If it is blocking the main thread, the Editor would freeze too until the loading or download is complete. It's simply waiting for www.SendWebRequest() to finish and while waiting, other scripts in your project are still running normally and every frame.

The problem is that some people don't understand how www.isDone is used and when to use it. One example is this post and when people find such code, they run into issues.

Two ways to use UnityWebRequest:

1. Download or make a request then forget it until it's done. You do this when you don't need to know the status of the download such as the UnityWebRequest.downloadedBytes, UnityWebRequest.uploadedBytes and UnityWebRequest.downloadProgress and UnityWebRequest.uploadProgress values.

To do this, you yield the UnityWebRequest.SendWebRequest() function with the yield return keyword. It will wait on the UnityWebRequest.SendWebRequest() function until the download is complete but it's not blocking your program. The code in your Update function and other scripts should still be running. That wait is only done in your LoadFileAsync coroutine function.

Example (Notice the use of yield return www.SendWebRequest() and no UnityWebRequest.isDone):

using (var www = new UnityWebRequest(pathOrUrl))
{
    www.downloadHandler = new DownloadHandlerBuffer();
    yield return www.SendWebRequest();

    if (www.isHttpError || www.isNetworkError)
    {
        Debug.Log("Error while downloading data: " + www.error);
    }
    else
    {
        finishedReading(www.downloadHandler.text);
    }
}

2. Download or make a request then wait every frame until UnityWebRequest.isDone is true. You do this when you need to know the status of the download such as the UnityWebRequest.downloadedBytes, UnityWebRequest.uploadedBytes and UnityWebRequest.downloadProgress and UnityWebRequest.uploadProgress values and these can be check while waiting for UnityWebRequest.isDone to be true in a loop.

You cannot wait or yield the UnityWebRequest.SendWebRequest() function in this case since yielding it will wait there until the request is complete making it impossible to check the download status. Just call the UnityWebRequest.SendWebRequest() function like a normal function then do the waiting in a while loop with UnityWebRequest.isDone.

The waiting for a frame is done in a loop with yield return null or yield return new WaitForEndOfFrame() but it's recommended to use yield return null since that doesn't create GC.

Example (Notice the use of UnityWebRequest.isDone and no yield return www.SendWebRequest()):

using (var www = new UnityWebRequest(pathOrUrl))
{
    www.downloadHandler = new DownloadHandlerBuffer();
    //Do NOT yield the SendWebRequest function when using www.isDone
    www.SendWebRequest();

    while (!www.isDone)
    {
        updatePerc.Invoke(www.downloadedBytes / length); // currentLength / length
        yield return null;
    }

    if (www.isHttpError || www.isNetworkError)
    {
        Debug.Log("Error while downloading data: " + www.error);
    }
    else
    {
        finishedReading(www.downloadHandler.text);
    }
}

You are mixing both #1 and #2. In your case, you need to use #2. Notice in both cases, I checked for error after the downloading and before using the loaded data with if (www.isHttpError || www.isNetworkError). You must do this.

---

The WWW class I used (it will be deprecated in a future) doesn't block the main thread, but it only displays two steps on the progress bar (like 25% and 70%).

If that's the true, then it is likely that after fixing the issue I talked about with UnityWebRequest, the UnityWebRequest API will likely give you 25% and 70% like the WWW API too.

The reason you see 25% and 70% is because the file size is very small so the Unity API is loading it quickly that it skips some percentage values. This is normal with the Unity's API. Just use any C# System.IO API to read the file to workaround this. Make your ThreadedFileRead function return the result via Action just like your LoadFileAsync function. This will make it easier to implement the Thread.

Grab the UnityThread script which is used to make a callback to the main Thread. You do this so that you can use the values returned from the callback with the Unity API on the main Thread.

Initialize the Thread callback script in the Awake function then use ThreadPool.QueueUserWorkItem or Task.Factory.StartNew to handle the file loading. To send the value back to the main Thread, use UnityThread.executeInUpdate to make the call.

void Awake()
{
    UnityThread.initUnityThread();
}

public static void ThreadedFileRead(string path, Action<float> percAction, Action<string> finishedReading)
{
    /* Replace Task.Factory with ThreadPool when using .NET <= 3.5
     * 
     * ThreadPool.QueueUserWorkItem(state =>
     * 
     * */

    var task = Task.Factory.StartNew(() =>
    {
        FileInfo fileInfo = new FileInfo(path);
        StringBuilder sb = new StringBuilder();

        float length = fileInfo.Length;
        int currentLength = 0;

        using (StreamReader sr = new StreamReader(path))
        {
            while (!sr.EndOfStream)
            {
                string str = sr.ReadLine();
                sb.AppendLine(str);

                // yield return str;

                //Call on main Thread
                UnityThread.executeInUpdate(() =>
                 {
                     percAction(currentLength / length);
                 });

                currentLength += str.Length;
                //Interlocked.Add(ref currentLength, str.Length);
            }

            //Call on main Thread
            UnityThread.executeInUpdate(() =>
             {
                 finishedReading(sb.ToString());
             });
        }
    });
}

Usage:

ThreadedFileRead(path, (percent) =>
{
    Debug.Log("Update: " + percent);
}, (result) =>
{
    Debug.Log("Done: " + result);
});