3

I have an architecture that reads packets from a packetized binary file, assigns each packet to an individual processing pipeline based on the packet type, and reassembles the packets into a new file on the other side of the pipelines. Each pipeline contains blocking queues similar to this one.

There is a thread on each side of the blocking queue in each pipline that runs a loop that queues or dequeues packets. These threads are started asynchronously (i.e. "fire and forget" style) from a controller object. This controller object has a Dictionary<int, ChannelPipeline> collection that contains all of the pipeline objects.

Here's my question: What mechanism can I put in place that will tell me when all of the pipelines have completed processing? There's an EndOfData property on each pipeline; do I have to continuously poll that property on every pipeline until they all read true, or is there a better (i.e. more efficient) way?

Community
  • 1
  • 1
Robert Harvey
  • 178,213
  • 47
  • 333
  • 501
  • 1
    Make `EndOfData` an `AutoResetEvent` and `WaitHandle.WaitAll()` is your friend. – Hans Passant Aug 12 '11 at 16:45
  • @Hans Passant: Unless he has more than 64 pipelines (or WaitHandles for that matter), [WaitAll()](http://msdn.microsoft.com/en-us/library/z6w25xa6.aspx) doesn't seem to allow more handles to be waited on at once. – Christian.K Aug 12 '11 at 16:51

1 Answers1

2

If you know the total number of pipelines you can consider AutoResetEvent + integer counter

AutoResetEvent allThreadsDone = new AutoResetEvent(false);
int completedThreads;
int scheduledThreads;

basically each worker thread will increment it's value used Interlocked.Increment() and set event in case when it is last thread:

Interlocked.Increment(ref completedThreads);
if (completedThreads == scheduledThreads)
{
    allThreadsDone.Set();
}

In monitoring thread you just do:

 allThreadsDone.WaitOne();
   // here is we know that all threads are finished
sll
  • 61,540
  • 22
  • 104
  • 156
  • No, that only stops threads from completing. A semaphore counts the wrong way. – Hans Passant Aug 12 '11 at 16:44
  • @Hans Passant : Yep, I fixed that and basically removed an idea using Semaphore at all because not see way of handling that Semaphore is full/empty – sll Aug 12 '11 at 16:47
  • Just to add, if you're using .NET 4.0, you can also use the [CountdownEvent class](http://msdn.microsoft.com/en-us/library/dd235708.aspx), which seems to provide just that functionality. – Christian.K Aug 12 '11 at 16:54