Threading lists of tasks in .NET 4

Question

I have a set of nicely structured objects set up along these lines:

abstract class Data 
{
  public abstract void GetData();
  public abstract void Process();
}
abstract class DataSource1 : Data
{
  public void GetData() { ... }
}
class DataSource1ProcessorA : DataSource1
{
  public void Process() { ... }
}
abstract class DataSource2 : Data
{
  public void GetData() { ... }
}
class DataSource2ProcessorA : DataSource2
{
  public void Process() { ... }
}
class DataSource2ProcessorB : DataSource2
{
  public void Process() { ... }
}

Currently, I'm processing each DataSourceProcessor sequentially, which is starting to get long as we add more sources and processors. Each source is completely unrelated, but each processor needs to be run in order. So I need a way to turn

new List<List<Data>> tasks = new List<List<Data>>()
{
  new List<Data>() { new DataSource1ProcessorA() },
  new List<Data>() { new DataSource2ProcessorA(), new DataSource2ProcessorB() }
}
foreach (List<Data> source in tasks)
{
  foreach(Data data in source)
  {
    data.GetData();
    data.Process();
  }
}

into a series of Tasks. I've worked with Tasks before, although I haven't done anything really complicated with them, and I just can't figure out how to fire off an arbitrary number of tasks with an arbitrary sequence of sequential calls.

I don't need to access the Tasks in any way, other than kicking them off and reporting any errors they throw. Preferably an error in DataSource2ProcessorA shouldn't prevent DataSource2ProcessorB from running, but if the coding is significantly easier to allow one error to kill the whole DataSource's thread, I can live with that.

Teoman Soygul · Answer 1 · 2011-05-23T19:22:43.057

2

Use below to fire off each processing event on a separate thread and log the errors (or re-throw exceptions):

Task.Factory.StartNew(() => { data.GetData(); data.Process(); })
.ContinueWith(t => Logger.Error("An exception occurred while processing. Check the inner exception for details", t.Exception),
TaskContinuationOptions.OnlyOnFaulted);

edited May 23 '11 at 19:22

answered May 23 '11 at 19:13

Teoman Soygul

25,584
6
69
80

score 1 · Answer 2 · edited May 23 '17 at 12:04

1

see Multithreading improvements in .NET 4

.net 4 added the ability to do parallel foreach.

edited May 23 '17 at 12:04

Community

1
1

answered May 23 '11 at 19:14

Fourth

9,163
1
23
28

score 1 · Accepted Answer · answered May 23 '11 at 19:28

You want to use a parallel foreach, and an action that processes the list in order for each item. That is:

new List<List<Data>> tasks = new List<List<Data>>()
{
  new List<Data>() { new DataSource1ProcessorA() },
  new List<Data>() { new DataSource2ProcessorA(), new DataSource2ProcessorB() }
}

Parallel.ForEach(tasks, (items) =>
{
    foreach (var item in items)
    {
        item.GetData();
        item.Process();
    }
});

That that, although this ensures that DataSource2ProcessorA is processed before DataSource2ProcessorB, there's no guarantee about the order in which tasks are run.

I knew that there was probably some way to do this with Parallel, but I was thinking in terms of nested Parallels and ContinueWith actions. I never considered just using the "old school" foreach loop inside. (Nor was I quite sure how to pass the list in.) Thanks! — Bobson, May 23 '11 at 19:35

score 0 · Answer 4 · answered May 23 '11 at 19:13

0

Why don't you use parallel.FX library? you can use the parallel.For or Parallel.ForEach methods easily. http://msdn.microsoft.com/en-us/magazine/cc163340.aspx

answered May 23 '11 at 19:13

Alireza

5,421
5
34
67

Threading lists of tasks in .NET 4

4 Answers4