Parallel.For vs for

Question

I have a Parallel.For and a regular for loop doing some simple arithmetic, just to benchmark Parallel.For

My conclusion is that, the regular for is faster on my i5 notebook processor.

This is my code

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            int Iterations = int.MaxValue / 1000;
            DateTime StartTime = DateTime.MinValue;
            DateTime EndTime = DateTime.MinValue;

            StartTime = DateTime.Now;
            Parallel.For(0, Iterations, i =>
            {
                OperationDoWork(i);
            });
            EndTime = DateTime.Now;
            Console.WriteLine(EndTime.Subtract(StartTime).ToString());

            StartTime = DateTime.Now;
            for (int i = 0; i < Iterations; i++)
            {
                OperationDoWork(i);
            }
            EndTime = DateTime.Now;
            Console.WriteLine(EndTime.Subtract(StartTime).ToString());

            StartTime = DateTime.Now;
            Parallel.For(0, Iterations, i =>
            {
                OperationDoWork(i);
            });
            EndTime = DateTime.Now;
            Console.WriteLine(EndTime.Subtract(StartTime).ToString());

            StartTime = DateTime.Now;
            for (int i = 0; i < Iterations; i++)
            {
                OperationDoWork(i);
            }
            EndTime = DateTime.Now;
            Console.WriteLine(EndTime.Subtract(StartTime).ToString());
        }

        private static void OperationDoWork(int i)
        {
            int a = 0;
            a += i;
            i = a;
            a *= 2;
            a = a * a;
            a = i;
        }
    }
}

And these are my results. Which on repetition do not change much:

00:00:03.9062234
00:00:01.7971028
00:00:03.2231844
00:00:01.7781017

So why ever use Parallel.For ?

If memory serves, `Parallel.For` doesn't always break the work down into multiple threads, it might run the entire thing on a single thread. You might be viewing the overhead involved in using the `Parallel` methods on lightweight pieces of work, where it is costly to marshal work over threads. — Adam Houldsworth, May 31 '13 at 12:10
The actual work you do in each iterartion is not that much, so the cost of creating and managing and consolidating threads is much higher than the gain you get from parallel proccessing. `Parallel.For` will only be faster when you do a lot more time consuming stuff each round. — Corak, May 31 '13 at 12:13
[Have a look at this recent answer where I make some detailed timings](http://stackoverflow.com/a/16822242/106159). `Parallel.For()` does work well for small loop bodies if you use a `Partitioner` as I do in that answer. — Matthew Watson, May 31 '13 at 12:32

score 11 · Answer 1 · answered May 31 '13 at 12:15

11

Parallel processing has organization overhead. Think of it in terms of having 100 tasks and 10 people to do them. It's not easy to have 10 people working for you, just organizing who does what costs time in addition to actually doing the 100 tasks.

So if you want to do something in parallel, make sure it's so much work that the workload of organizing the parallelism is so small compared to the actual workload that it makes sense to do it.

answered May 31 '13 at 12:15

nvoigt

75,013
26
93
142

3

Very nice explanation. "I'm faster doing it by myself!" is probably an experience everyone had at least once with "teamwork". – Corak May 31 '13 at 12:19

score 7 · Accepted Answer · edited Jun 20 '20 at 09:12

One of the most common mistakes one does, when first delving into multithreading, is the belief, that multithreading is a Free Lunch.

In truth, splitting your operation into multiple smaller operations, which can then run in parallel, is going to take some extra time. If badly synchronized, your tasks may well be spending even more time, waiting for other tasks to release their locks.

As a result; parallelizing is not worth the time/trouble, when each task is going to do little work, which is the case with OperationDoWork.

Edit:

Consider trying this out:

    private static void OperationDoWork(int i)
    {
        double a = 101.1D * i;
        for (int k = 0; k < 100; k++)
            a = Math.Pow(a, a);
    }

According to my benchmark, for will average to 5.7 seconds, while Parallel.For will take 3.05 seconds on my Core2Duo CPU (speedup == ~1.87).
On my Quadcore i7, I get an average of 5.1 seconds with for, and an average of 1.38 seconds with Parallel.For (speedup == ~3.7).

This modified code scales very well to the number of physical cores available. Q.E.D.

Parallel.For vs for

2 Answers2

Edit: