What are the guidelines for Parallel.ForEach vs. foreach?

Question

I'm curious about the performance characteristics of Parallel.ForEach. Given any valid construct inside a Parallel.ForEach loop, is it always preferable to use Parallel.ForEach over a foreach loop? I'm specifically wondering about the overhead of invoking the Parallel Tasks library on small sets or other edge cases where a foreach loop might be faster. I know the library is pretty smart about when/how to spawn threads...are there cases where it's better to just leave code in a foreach loop, or is the overhead for calling Parallel Tasks generally negligble, so if you can, you should use Parallel.ForEach?

This question is similar and provides good functional difference information but doesn't really speak to performance. Note that I'm ignoring compatibility to .NET <4 as a reason for staying with a foreach:

C#: Any benefit of List<T>.ForEach(...) over plain foreach loop?

Answer 1

It's not always preferable. For fast loop bodies, Parallel.ForEach can degrade performance. One of the guidelines listed in Parallel Programming Coding Guidelines is to measure both before and after parallelization.

Other useful articles have been published by the parallel computing group.

Answer 2

I would say to always stick with simple (i.e. regular foreach) and only implement more complex stuff (i.e. Parallel ForEach) when you have a measurable requirement to do so. So if you can prove that in a particular instance that the normal foreach isn't as fast as you need, and you can prove for that particular instance that the Parallel foreach will solve your problem, then use the Parallel.

Otherwise just keep it simple.