8

I'm toying with the idea of implementing a generic Producer/Consumer pair + processing queue in C# for fun. The idea is you can just create objects that implement appropriate IProducer and IConsumer interfaces (default implementations supplied), which will consist mainly of delegates, pass them to a QueueProcessor class instance, tell it how many consumers you want, and go.

But I say to myself, "Self, surely this has been done before."

So does anyone know of a good, generic implementation of the producer/consumer pattern in C# (VB.Net is okay, too)? The basic requirements I'm looking for:

  • Use generics for the produced and consumed types (input, queued task, and output types, or any combination thereof)
  • Allow you specify how many consumers will work the queue
  • Allow you to link up or chain multiple queues into a pipeline (tricky with multiple Consumers, I know)
  • Allow you to implement your own Producers and Consumers
  • Allow you to turn any IEnumerable into a producer (okay if I have to implement that myself, as long it's possible)
  • Delegate based (you can use lambda syntax for the basic consumer or producer work to process a single item)

Or if there is none, what pitfalls have prevented it and do you have any thoughts on how to implement it?

Joel Coehoorn
  • 399,467
  • 113
  • 570
  • 794
  • Have you looked at http://msdn.microsoft.com/en-us/library/dd267312.aspx ? – porges Sep 13 '11 at 20:39
  • Yes, and IProducerConsumerCollection, but none of that existed yet when I first asked the question. Write up a post about the support in 4.0 and you'll get an easy upvote from me. – Joel Coehoorn Sep 13 '11 at 21:05
  • Argh! I don't know how this keeps happening... I'm always ending up on 2-year old posts :P – porges Sep 13 '11 at 21:14
  • Hi Joel Coehoorn, please take a look at this [Producer-Consumer example](https://stackoverflow.com/questions/733793/implementing-the-producer-consumer-pattern-in-c-sharp/47179576#47179576) which uses BlockingCollection. It may be useful for building your Producer-Consumer classes in case you go for building your own Producer-Consumer framework. – sɐunıɔןɐqɐp Nov 08 '17 at 13:33

3 Answers3

5

Microsoft CCR contains much of what you need.

Here are some code samples and usage notes.

Scott Fraley
  • 378
  • 4
  • 14
ripper234
  • 222,824
  • 274
  • 634
  • 905
  • It is not a perfect framework though, lacking some obvious stuff like the ability to STOP the processing and WAIT for remaining tasks. We are enhancing the framework in our dev team, if you need more help let me know... – ripper234 Jun 03 '09 at 08:03
2

Marc Gravell wrote a nice example blocking queue in this answer.

Community
  • 1
  • 1
Don Kirkby
  • 53,582
  • 27
  • 205
  • 286
0

Have you looked at MiscUtil ?

Ruben Bartelink
  • 59,778
  • 26
  • 187
  • 249