Parallel.ForEach working too slow for DataTable

Question

I have a problem implementing Parallel.ForEach in DTable. Following is the sample code that I am implementing:

Parallel.ForEach(dTable4.AsEnumerable(), row4 =>
            {
                string getCondition = row4[1].ToString();
                getCondition = EvaluateCondExpression(getCondition);  //Evaluates CM for value (CM1==2 && CM2<5);
                Expression e = new Expression(getCondition); //getCondition = (2==2 && 2<5)
                var evalutateCondition = e.Evaluate();
                if (Convert.ToBoolean(evalutateCondition))
                {
                   //Write string to TextBox
                }
             }

When I run this, the threads are not efficiently managed and it takes far too time than foreach loop. EvaluateCondExpression function returns value after checking the user provided parameter from GUI combobox and numericUpDown

private string EvaluateCondExpression(string getCondition)
    {
        string[] splitgetCondition1 = SeprateCharacter(getCondition);
        foreach (string oneCondition in splitgetCondition1)
        {
            string conditionValue = oneCondition;
            if (oneCondition.Contains("CM"))
            {
                conditionValue = RemoveMultipleChar(conditionValue.Replace('!', ' ').Trim());
                string getInputNumber = getTableOneInputNo(conditionValue, dTable1);
                string tableOneValue = checkComboxValue(m_comboBox, getInputNumber);

                getCondition = getCondition.Replace(conditionValue, tableOneValue);
            }
        }
        return getCondition;
    }

The serial ForEach computation is taking too much time so I wanted to apply Parallel.ForEach iteration but unfortunately it is not working. Can anyone suggest me how to maximize the performance and what I am doing wrong in Parallel.ForEach.

You might see a performance increase by using a traditional foreach, wrapping each iteration in a Task. Add the task for each iteration to a collection, then outside of the foreach call `Task.WhenAll(tasks);` if anything it gains you the ability to await the expensive parallel process. — Johnathon Sullinger, Jul 01 '15 at 04:59
Reference showing a performance improvement? Or links showing an example of doing it that way? — Johnathon Sullinger, Jul 01 '15 at 05:31
For long running tasks, anything beyond 2-3 sec consider using Async - Await, instead of Parallel.ForEach. Infact Async - Await is not even blocking like Parallel call — Mrinal Kamboj, Jul 01 '15 at 06:37

score 3 · Answer 1 · answered Jul 01 '15 at 05:55

You might see a performance increase by using a traditional foreach, wrapping each iteration in a Task. Add the task for each iteration to a collection, then outside of the foreach call Task.WhenAll(tasks); if anything it gains you the ability to await the expensive parallel process.

You can convert the contents of your Parallel.ForEach in to a Select Linq query that turns each iteration in to a Task. The resulting collection of Tasks can be given to the Task.WhenAll method to await

await Task.WhenAll(dtable4.AsEnumerable().Select(row4 => new Task(() =>
{
    string getCondition = row4[1].ToString();
    getCondition = EvaluateCondExpression(getCondition);  //Evaluates CM for value (CM1==2 && CM2<5);
    Expression e = new Expression(getCondition); //getCondition = (2==2 && 2<5)
    var evalutateCondition = e.Evaluate();
    if (Convert.ToBoolean(evalutateCondition))
    {
        //Write string to TextBox
    }
})));

This might not solve the performance bottle neck you're seeing, but this will at least let you await the parallel process and free up the UI thread.

This is a good idea, you may check if degree of parallelism helps, else this would be a way to go, especially if you need call to be non blocking — Mrinal Kamboj, Jul 01 '15 at 06:47
He should also consider not writing to the UI until the `WhenAll` is completed, so each task isn't switching to the UI sync context to write to it. Writing to the UI is expensive if you are using any binding as all of the bindings have to be refreshed. He didn't specify his UI framework so the OP will have to use his discretion — Johnathon Sullinger, Jul 01 '15 at 14:46

score 2 · Answer 2 · edited May 23 '17 at 12:14

2

How long does it take to run one iteration of the loop? How many iterations are you running? In general, Parallel.Foreach() is well suited to running through loops where each iteration potentially takes quite a while. If you instead have many iterations of a relatively fast operation, you will spend a lot of extra overhead generating and managing threads, which is probably what you're seeing. See this MSDN article for more information on this type of scenario:

https://msdn.microsoft.com/en-us/library/dd560853(v=vs.110).aspx

Update

Here is a good comparison of Parallel.Foreach() and await Task.WhenAll().

edited May 23 '17 at 12:14

Community

1
1

answered Jul 01 '15 at 03:22

CrumblyRock

371
3
14

Size of iteration depends upon user selection parameters but for testing I am considering 7 loops for the first Parallel.ForEach function which is taking about 5596ms to run in serial ForEach operation and Parallel.ForEach is taking about 12879ms to perform the same task. I want to boost the performance as it will lag exponentially when dealing with many iteration. Iteration size may go about 20~25 loops which may force user to wait for more then 15~20Sec. – sk3145 Jul 01 '15 at 05:34
Parallel.Foreach is not good for long running tasks as suggested, in fact if each Parallel thread has a long duration task, then it will lead to more context switching and contention, as they would keep core busy for longer duration, when other threads are trying for it, Async - Await is better option for a long running task, in fact it is not even blocking like Paralel.ForEach – Mrinal Kamboj Jul 01 '15 at 06:36
1

@sk3145 you may have to spend some time doing some benchmarking. Try following the example in the MSDN link I provided. As in the other answer, Parallel.ForEach() has several options that can be used to adjust how many threads it spawns and a handful of other things, all of which you can find by browsing around the MSDN article. It is difficult to say exactly what would make it perform the best without trying different things out. – CrumblyRock Jul 01 '15 at 13:01

Mrinal Kamboj · Answer 3 · 2015-07-01T06:57:07.087

You can make the following modification to the code, when using Parallel.ForEach, especially for the long running tasks

Parallel.ForEach( dTable4.AsEnumerable(), new ParallelOptions {MaxDegreeOfParallelism = Environment.ProcessorCount}, row4 = >

This would ensure that Parallel.ForEach will not start a thread of each data point in the datatable, it will only spawn the threads based on number of environment processors / cores dynamically, thus will reduce the contention and thread context switching. However ideally as suggested above you may plan Asynchronous pattern for long running tasks.

As I see you are updating the UI control inside Parallel loop, for that you anyway need UI thread context, else it will an exception. Async anyway run in UI thread context

Parallel.ForEach working too slow for DataTable

3 Answers3