Parallel.ForEach - First thread runs slower than the rest

Question

This is a strange issue that I couldn't find much information about from searches.

I have a listview control with a list of video files in it. I use Parallel.ForEach to run separate threads to process each video file frame by frame for motion. Each loop through frames includes some invokes to update controls on the form which I assume the main thread (ID 1) handles.

What is happening is that the first file to be processed seems to take much longer than the rest to complete, in fact, on low resolution video files, that thread hangs completely while the rest zoom through the frames. Occasionally that thread will not complete at all, while the rest run though the rest of the files.

I believe this may be caused if the main thread is being used for processing, and gets held up by the invokes from the other threads.

EDIT: I have noticed that the thread hangs completely when another form is open.

Here is a screenshot of the application, showing the first thread running behind the rest

The code

 Parallel.ForEach<ListViewItem>(filesListView.Items.Cast<ListViewItem>(), new ParallelOptions() { MaxDegreeOfParallelism = Convert.ToInt32(Math.Ceiling((Environment.ProcessorCount * 0.75) * 1.0)) }, (item, state) =>
     {
         Thread.Sleep(100);
             if (CallToStop == true)
             {
                 state.Break();
             }
             internalProcessStart(item);
      });

Did you try to remove invokes at all, and check if this related? — Zergatul, Oct 02 '16 at 18:16
@TadijaBagarić Not file size related, I tried with the same file copied over and over and it happens there too. — user2924019, Oct 02 '16 at 18:35
@Zergatul Not yet but I will do this. I'm sure it must be the invokes though but will double check. — user2924019, Oct 02 '16 at 18:36
Show us how and where you call the ForEach. That " hangs when another form is open" indicates you call it from the main thread. You shouldn't. — H H, Oct 02 '16 at 18:38
@Zergatul I took out all of the invokes apart from one which I needed in order to see if the file had been processed and it was so much quicker, but was still slightly slow — user2924019, Oct 02 '16 at 18:41
@HenkHolterman The ForEach is called from another function which is called when a button is pressed on the main form. — user2924019, Oct 02 '16 at 18:43
@HenkHolterman You gave me something to go on there. Apparently launching the foreach from the UI is a big no no! Just need to figure out how to launch it properly http://stackoverflow.com/questions/8365346/why-does-this-parallel-foreach-code-freeze-the-program-up — user2924019, Oct 02 '16 at 18:45
Your code should already be thread-safe, so just call the ForEach from a pool thread (Backgroundworker). — H H, Oct 02 '16 at 18:56
@HenkHolterman I'm fairly sure it's thread safe, I've done a huge amount of testing but this is the only issue I am hitting. I have found this code which should be added before the `Parallel.ForEach`: `Task.Factory.StartNew(() =>" however hitting exceptions. I will continue to look at this. — user2924019, Oct 02 '16 at 19:00
@HenkHolterman, but if it was called from UI thread, shouldnt this freeze UI completely? In this case UI seems not frozen and is updating. — Evk, Oct 02 '16 at 19:31
I noticed it updates very slowly. For example I have a timer which is set to 1000ms which outputs the total frames processed by all the threads to a label. This freezes altogether until another thread finishes. The UI thread seems to do some processing when the other threads are running other tasks that don't require the UI to be updated. — user2924019, Oct 02 '16 at 19:36
@Evk - we don't see the complete code, there is mention of Invokes. The messageloop seems to be at least partially running. — H H, Oct 02 '16 at 19:37
The UI does freeze entirely when the threads have reached the last items to process, but comes back to life when all have finished. Forgot to mention. — user2924019, Oct 02 '16 at 19:40

score 0 · Answer 1 · answered Oct 02 '16 at 19:26

With help from Henk Holterman in the comments, I found the reason is due to the Parallel.ForEach being called from the UI thread when it should be a background worker thread in order to keep the UI thread free.

I will update this answer when I have the correct code working with my code.

Parallel.ForEach - First thread runs slower than the rest

1 Answers1