Parallel.For Support for writing to arrays?

Question

Here is some code I found on the MSDN website.

long[] nums = { 1, 2, 3, 4 };
long total = 0;
Parallel.For<long>(0, nums.Length, () => 0, (j, loop, subtotal) =>
{
    subtotal += nums[j];
    return subtotal;
}, (x) => Interlocked.Add(ref total, x));

Im wondering if somebody could show me how to modify this code to write the results back to a shared array? Something like below.

My_Class[] localArray;
Parallel.For<My_Class[]>(0, localArray.Length, () => 0, (j, loop, messages)  =>
{
    List<My_Class> tempList = new List<My_Class>();
    My_Class myClass = localArray[j];
    foreach (Message msg in myClass.GetListOfMessages())
    {
         if (msg.getHeader() == "Hello")
         {
              tempList.add(msg);
         }
    }
    messages = temp.toArray();
    return messages;
}, (x) => Interlocked.Add(ref localArray, x));

So you see the example above I have a shared local array which contains objects of type "My_Class". The My_Class object contains a list of messages of which I want to search to find specific types of message with headers "Hello".

So in the example code I use the parallel for loop to iterate through my custom objects and create a temporary list called "tempList" which I eventually convert to an array and return as a result.

However the code above just doesn't work for arrays, specifically the (x) => Interlocked line, it doesn't seem to be compatible with merging the results returned by many arrays into one array.

Please can someone assist me.

Any shared data is a problem for parallel processing. In your case though, you don't even need it. You could easily use a ConcurrentQueue instad of an array or list. Moreover, the MSDN example addresses a *completely* different scenario than yours. MSDN deals with parallel processing of a lot of data, partitioning and collecting the result. Your code is nothing of the sort - it's processing individual messages. There's no need for a final array or Interlocked at all — Panagiotis Kanavos, Jun 01 '16 at 07:55

score 0 · Answer 1 · answered May 31 '16 at 10:40

0

Interlocked only support two Add methods:

Add(Int32, Int32) 
// Adds two 32-bit integers and replaces the first integer with the sum, as an atomic operation.

Add(Int64, Int64)   
// Adds two 64-bit integers and replaces the first integer with the sum, as an atomic operation.

So I don't think it will work with some kind of object.

answered May 31 '16 at 10:40

wake-0

3,918
5
28
45

What would a possible solution be? – edb500 May 31 '16 at 10:41
Have you checked this post: [link](http://stackoverflow.com/questions/8001748/parallel-foreach-with-adding-to-list) Use a lock object or a thread safe list. – wake-0 May 31 '16 at 10:44
Maybe the last posts of this article are also interesting: [link](http://stackoverflow.com/questions/8001748/parallel-foreach-with-adding-to-list) – wake-0 May 31 '16 at 10:48
This should be a comment, not an answer. In any case, Interlocked in the MSDN example is used for summing. What the OP asks has nothing to do with sums or aggregations. – Panagiotis Kanavos Jun 01 '16 at 08:09

score 0 · Answer 2 · answered Jun 01 '16 at 08:17

You are trying to use an inappropriate example.

The MSDN code demonstrates numeric processing over a lot of data to produce a single result by partitioning the data and producing subtotals, then collecting the subtotals into a final result. The second step is performed by Interlocked.Add, because the number of subtotals will be very small.

Your code does something completely different, filter/process a large number of messages without trying to aggregate the partial results.

The simplest way to find the messages would be to use PLINQ and Where:

var helloMessages=from msgClass in  localArray.AsParallel()
                  from msg in msgClass.GetListOfMessages()
                  where msg.getHeader()=="Hello"
                  select msg;
var results=helloMessages.ToArray();

The only difference from a pure LINQ query is the AsParallel() keyword. The same can be written using extension methods directly:

var fesults=localArray.AsParallel()
                      .SelectMany(mscClass=>msgClass.GetListOfMessages())
                      .Where(msg=>msg.getHeader()=="Hello")
                      .ToArray();

You should probably use properties instead of getter methods, if only to make the code cleaner.

var fesults=localArray.AsParallel()
                      .SelectMany(mscClass=>msgClass.ListOfMessages)
                      .Where(msg=>msg.Header=="Hello")
                      .ToArray();

If for some reason you want to write data to a collection, you should use one of the concurrent collections like ConcurrentQueue, eg ConcurrentQueue<Message>:

ConcurrentQueue<Message> results=new ConcurrentQueue<Message>();
Parallel.ForEach(longMessageCollection,msg=>
        {
            //Do some really complex processing to select the message
            results.Enqueue(msg);
        });

Panagiotis, you're a god damn genius. Thanks for the help! :) — edb500, Jun 01 '16 at 09:33

Parallel.For Support for writing to arrays?

2 Answers2