Parallel.ForEach takes more time than expected

Question

Here the code on this concern:

while (true)
{
    Console.WriteLine("start " + DateTime.Now);
    ParallelOptions options = new ParallelOptions();
    options.MaxDegreeOfParallelism = -1;                
    Parallel.ForEach(hosts, item =>
    {
        using (Ping ping = new Ping())
        {
            PingReply pingReply = ping.Send(item.Value, 2000);  // timeout is 2 secs
            App.dict[item.Key].lastConnectTry = new KeyValuePair<bool, DateTime>((pingReply.Status == IPStatus.Success), DateTime.Now);
        }
    });        
    Console.WriteLine("end " + DateTime.Now);
    Thread.Sleep(15000);
}

But, then I run that app it gives slightly different results:

start 27.04.2012 10:12:32
end 27.04.2012 10:12:42
// it took 10 seconds
start 27.04.2012 10:12:57
end 27.04.2012 10:13:02
// this took 5 secs
start 27.04.2012 10:13:17
end 27.04.2012 10:13:22
//   5 secs
start 27.04.2012 10:13:37
end 27.04.2012 10:13:42
// 5 secs
start 27.04.2012 10:13:57
end 27.04.2012 10:14:01
start 27.04.2012 10:14:16
end 27.04.2012 10:14:19
start 27.04.2012 10:14:34
end 27.04.2012 10:14:36
start 27.04.2012 10:14:51
end 27.04.2012 10:14:54
start 27.04.2012 10:15:09
end 27.04.2012 10:15:11
start 27.04.2012 10:15:26
end 27.04.2012 10:15:29
start 27.04.2012 10:15:44
end 27.04.2012 10:15:46
start 27.04.2012 10:16:01
end 27.04.2012 10:16:06
start 27.04.2012 10:16:21
end 27.04.2012 10:16:24
start 27.04.2012 10:16:39
end 27.04.2012 10:16:41
start 27.04.2012 10:16:56
end 27.04.2012 10:16:59
start 27.04.2012 10:17:14
end 27.04.2012 10:17:16

Seems like it takes some time to bootstrap the threads, creating them and then Parallel execution time will arranged correctly.

So the question is why it takes more than 2 secs to handle all processing and how can I avoid it by bootstrap the threads before processing?

Update:

This loop is placed in separate background thread.

How many elements in `hosts`, and how many cores do you have? And what do you mean by "boostrapping" the threads? — Cameron, Apr 27 '12 at 04:18
hosts are filled by 22 elements. Two cores. "Bootstrapping" means kinda creating threads in threadpool that being used. I'm not so clear about that. — kseen, Apr 27 '12 at 04:19

score 2 · Accepted Answer · edited May 23 '17 at 12:22

2

Yes, threadpool needs to grow number of threads based on load (unrelated to Parallel.ForEach).

Check Parallel.ForEach not spinning up new threads and Parallel.Foreach spawning way too many threads for some background.

edited May 23 '17 at 12:22

Community

1
1

answered Apr 27 '12 at 04:28

Alexei Levenkov

98,904
14
127
179

So how can I make this loop always use concrete count of threads? – kseen Apr 27 '12 at 04:39
I don't think so, you can restrict max, but otherwise ForEach and ThreadPool will try to adjust to your load. – Alexei Levenkov Apr 27 '12 at 04:44
2

You can try to pre-grow the threadpool (ThreadPool.SetMinThreads) as discussed http://stackoverflow.com/questions/9248199/threadpool-behaviour-not-growing-from-minimum-size – Alexei Levenkov Apr 27 '12 at 04:50
Thanks, Alexei! Pre-growing the threadpool nicely helped out. Spasibo, Alexei! – kseen Apr 27 '12 at 05:10

yamen · Answer 2 · 2012-04-27T18:56:13.370

You're not actually passing your parallel options through, see here:

ParallelOptions options = new ParallelOptions();
options.MaxDegreeOfParallelism = -1;                
Parallel.ForEach(hosts, options, item => // note options passed through here
      // etc

If this isn't creating enough, you can increase the threadpool:

System.Threading.ThreadPool.SetMinThreads System.Threading.ThreadPool.SetMaxThreads

But note that really unless you know you're going to get improvements I wouldn't touch this. Since you're doing a Ping in your task code, this could be the thing adding the variability between each run.

It gives the same result. Max threads is setted to huge value so there is no any issues. — kseen, Apr 27 '12 at 04:55

Parallel.ForEach takes more time than expected

2 Answers2