Using Parallel.ForEach to add to a new List

Question

What is the best way to use parallelization such as with Parallel.ForEach so that I can rapidly iterate a collection and add items to a new List without violating thread safety but using the performance gain of multiple server cores and plenty of RAM?

public List<Leg> FetchLegs(Trip trip)
{
    var result = new List<Leg>();

    try
    {
        // get days
        var days = FetchDays(trip);

        // add each day's legs to result
        Parallel.ForEach<Day>(days, day =>
        {
            var legs = FetchLegs(day);
            result.AddRange(legs);
        });

        // sort legs by scheduledOut
        result.Sort((x, y) => x.scheduledOut.Value.CompareTo(y.scheduledOut.Value));

    }
    catch (Exception ex)
    {
        SiAuto.Main.LogException(ex);
        System.Diagnostics.Debugger.Break();
    }

    return result;
}

result.AddRange(legs) in current form is not thread safe, since List is not a thread safe structure, use something like ConcurrentQueue inside parallel loop, which can be converted to a List using ToList(). Otherwise the parallel code looks fine — Mrinal Kamboj, Jul 04 '15 at 18:54

Dzienny · Accepted Answer · 2015-07-04T20:26:29.367

3

The simplest solution is to use PLINQ and its extension methods like ToList, ToDictionary. Example:

var result = days.AsParallel().SelectMany(d => FetchLegs(d))
                 .OrderBy(l => l.scheduledOut.Value).ToList();

edited Jul 04 '15 at 20:26

answered Jul 04 '15 at 19:05

Dzienny

3,295
1
20
30

You need ForAll for action post AsParallel, since OP needs IEnumerable created post FetchLegs method to be added to result list using AddRange, current solution will lead to List> as final result, when expected is List – Mrinal Kamboj Jul 04 '15 at 19:23
and you could throw in an OrderBy() to replace the Sort() although I'm not sure if that will profit much from PLinq. – H H Jul 04 '15 at 19:41

score 2 · Answer 2 · edited May 23 '17 at 10:30

Shifting the comment to the answer with few more suggestions, beside using a thread safe structure like ConcurrentQueue for thread safety inside Parallel loop, or you may consider a custom thread safe list, check my following post - Thread safe List. In fact you may consider ReaderWriterSlim lock in place of plain lock. Another important point would be, you may consider using ParallelOptions MaxDegreeOfParallelism, check here with value equal to Environment.ProcessorCount, so that Parallel.ForEach partitioner works well, by not creating a task for each data point in the collection at the beginning, it will do it based on number of logical cores in the system, thus effectively reducing the thread context switching and making application much more efficient

Using Parallel.ForEach to add to a new List

2 Answers2