C# Slow UI Performance when calling BeginInvoke frequently

Question

I have a main form called ProxyTesterForm, which has a child form ProxyScraperForm. When ProxyScraperForm scrapes a new proxy, ProxyTesterForm handles the event by testing the scraped proxy asynchronously, and after testing adds the proxy to a BindingList which is the datasource of a DataGridView.

Because I am adding to a databound list which was created on the UI thread I am calling BeginInvoke on the DataGridView so the update happens on the appropriate thread.

Without the BeginInvoke call in the method I will post below, I can drag the form around on my screen during processing and it doesn't stutter and is smooth. With the BeginInvoke call, it's doing the opposite.

I have a few ideas on how to fix it, but wanted to hear from smarter people than me here on SO so I solve this properly.

1. Use a semaphore slim to control the amount of simultaneous updates.

2. Add asynchronously processed items to a list outside of the scope of the the method I will post below, and iterate over that list in a Timer_Tick event handler, calling BeginInvoke for each item in the list every 1 second, then clearing that list and wash, rinse, repeat until the job is done.

3. Give up the convenience of data binding and go virtual mode.

4. Anything else someone might suggest here.

   private void Site_ProxyScraped(object sender, Proxy proxy)
   {
       Task.Run(async () =>
       {
           proxy.IsValid = await proxy.TestValidityAsync(judges[0]);
           proxiesDataGridView.BeginInvoke(new Action(() => { proxies.Add(proxy); }));
       });
   }
   

Answer 1

In Windows every thread that has UI has a message queue - this queue is used to send UI messages for the windows for this thread, those message include things like mouse moved, mouse up/down, etc.

Somewhere in every UI framework there is a loop that reads a message from the queue, processes it and then wait for the next message.

Some messages are lower priority, for example the mouse move message is generated only when the thread is ready to process it (because the mouse tends to move a lot)

BeginInvoke also uses this mechanism, it send a message telling the loop there's code it needs to run.

What you are doing is flooding the queue with your BeginInvoke message and not letting it handle UI events.

The standard solution is to limit the amount of BeginInvoke calls, for example, collect all the items you need to add and use one BeginInvoke call to add them all.

Or add in batches, if you make just one BeginInvoke call per second for all the objects found in this second you probably not effect the UI responsiveness and the user won't be able to tell the difference.

Answer 2

Note: For the actual answer on why this is happening, see @Nir's answer. This is only an explanation to overcome som problems and to give some directions. It's not flawless, but it was in line of the conversation by comments.

Just some quick proto type to add some separation of layers (minimal attempt):

//member field which contains all the actual data
List<Proxy> _proxies = new List<Proxy>();

//this is some trigger: it might be an ellapsed event of a timer or something
private void OnSomeTimerOrOtherTrigger()
{ 
      UIupdate();
}

//just a helper function
private void UIupdate
{
    var local = _proxies.ToList(); //ensure static encapsulation 
    proxiesDataGridView.BeginInvoke(new Action(() => 
    {    
         //someway to add *new ones* to UI
         //perform actions on local copy
    }));
}

private void Site_ProxyScraped(object sender, Proxy proxy)
{
    Task.Run(async () =>
    {
        proxy.IsValid = await proxy.TestValidityAsync(judges[0]);
        //add to list
        _proxies.Add(proxy);
    });
}