Asynchronous method not adding frames to my List

Question

I have been battling to read frames from a video with the library Emgu CV and use a timer to play the frames in a picture box to create the impression of a video. This time round I used a rather sophisticated approach where I have an asynchronous method that Read Frames from the video and adds them to a List of Mat objects. I then subscribed the Form to the Application.Idle event handler which checks for me if the task object returned by the asynchronous method is not null and is completed. If those conditions are met then the timer is enabled which starts to loop through the Mat List and display them in the picture box at an interval of 33 miliseconds. However when I step through the debugger, the list of Mats is empty and it's count is 0. I have even done a Debug.WriteLine inside the asynchronous method to monitor what its doing but it's not adding Mat(which is basically an image container l) to my List. I have queried the frame count from the video using the library and the compiler returns 69 for the frame count which means the video has frames. Help me make this asynchronous method read frames into the list. Implementation details of the asynchronous method

private async Task ReadFrames(){
  if(video !=null){
   await Task.Run())=>{
     while(video.Grab()){
      //add the frames to the list
      frames.Add(video.QueryFrame());
      }
    });
  }
}

Inside my class I have defined a Task object, the list of frames, frame count and a timer object for playing the video.

public partial class Form1 :Form {
 //object to store the result of the asynchronous task
  private Task task = null;
  private Timer timer = new Timer();
  //integer to store the frame count
  private int frameCount = 0;
  //integer to keep track of the current frame 
  private int currentFrame = 0;
  //list to hold the frames in the video
  private List<Mat> frames = new List<Mat>();
  //inside the ctor, the timer is disabled by default
 public Form1(){
  timer.Interval = 33;
  timer.Enabled = false;
  timer.Elapsed += Timer_Elapsed;
 
  //subscribing to application idle event
  Application.Idle += Application_Idle;
 }
 //here I check if the task is completed and set to an instance 
   private void Application_Idle(object sender, EventArgs e){
    if(task !=null && task.IsCompleted) 
    { 
   //enable the timer to start looping through the frames
   timer.Enabled = true;
    }
  }
 }

Now the on elapse checks if the current frame index is less than the total and keeps changing at an interval of 33 milliseconds.

private void Timer_Elapsed(object? sender, ElapsedEventArgs e){
  if(currentFrame < frameCount){
    pictureBox1.Image = ToBitmap(frames[currentFrame];
    //Update the index
    currentFrame +=1;
    Task.Delay(33);
  }else{
    //disable the timer
     timer.Enabled = false:
  }
}

When I step through the debugger, it shows that frames has a count of 0, how can I fix this?

Answer 1

This is just not a good way to do asynchronous capture. It is needlessly complicated, and likely error prone. If you intend to capture some fixed number of frames you could do something like this:

public Task<List<Mat>> CaptureFrames(int frameCount){

    List<Mat> CaptureLocal(){
        var result = new List<Mat>();
        for(int i = 0; i < frameCount && video.Read(out var image); i++){
            result.Add(image);
        }
        return result;
    }
    return Task.Run(CaptureLocal);
}

To do playback you could do something like

public async Task Playback(IEnumerable<Mat> frames, CancellationToken cancel){
    foreach(var frame in frames){
        cancel.ThrowIfCancellationRequested();
        pictureBox1.Image = ToBitmap(frame);
        await Task.Delay(33);
    }
}

But note that timers in windows, including Task.Delay are limited to ~16ms by default. If you need better accuracy you might need to use MultiMediaTimer or increase the timer frequency during playback.

If you want concurrent capture and playback you can use ConcurrentQueue so that you can safely add frames from a background thread, and display them from the UI thread. But this can become complicated, and will depend on the priorities, i.e. should you prioritize smooth frame rate? or that the latest image is displayed? Using DataFlow may make things simpler.

Also note that there are a bunch of additional things you may need to consider for smooth and efficient playback. But using proper techniques for asynchronous code should at least help a bit.