Thread Pool of workers in a Window service?

Question

I'm creating a Windows service with 2 separate components: 1 component creates jobs and inserts them to the database (1 thread) The 2nd component processes these jobs (multiple FIXED # of threads in a thread pool)

These 2 components will always run as long as the service is running.

What I'm stuck on is determining how to implement this thread pool. I've done some research, and there seems to be many ways of doing this such as creating a class that overriddes the method "ThreadPoolCallback", and using ThreadPool.QueueUserWorkItem to queue a work item. http://msdn.microsoft.com/en-us/library/3dasc8as.aspx

However in the example given, it doesn't seem to fit my scenario. I want to create a FIXED number of threads in a thread pool initially. Then feed it jobs to process. How do I do this?

// Wrapper method for use with thread pool.

public void ThreadPoolCallback(Object threadContext)
{
    int threadIndex = (int)threadContext;
    Console.WriteLine("thread {0} started...", threadIndex);
    _fibOfN = Calculate(_n);
    Console.WriteLine("thread {0} result calculated...", threadIndex);
    _doneEvent.Set();
}

Fibonacci[] fibArray = new Fibonacci[FibonacciCalculations];
const int FibonacciCalculations = 10;

for (int i = 0; i < FibonacciCalculations; i++)
{
      ThreadPool.QueueUserWorkItem(f.ThreadPoolCallback, i);
}

score 1 · Answer 1 · edited May 23 '17 at 12:14

Create a BlockingCollection of work items. The thread that creates jobs adds them to this collection.

Create a fixed number of persistent threads that read items from that BlockingCollection and process them. Something like:

BlockingCollection<WorkItem> WorkItems = new BlockingCollection<WorkItem>();

void WorkerThreadProc()
{
    foreach (var item in WorkItems.GetConsumingEnumerable())
    {
        // process item
    }
}

Multiple worker threads can be doing that concurrently. BlockingCollection supports multiple readers and writers, so there's no concurrency problems that you have to deal with.

See my blog post Simple Multithreading, part 2 for an example that uses one consumer and one producer. Adding multiple consumers is a very simple matter of spinning up a new task for each consumer.

Another way to do it is to use a semaphore that controls how many jobs are currently being processed. I show how to do that in this answer. However, I think the shared BlockingCollection is in general a better solution.

Hmm that thread that creates jobs just inserts them to the db. The worker threads is supposed to read jobs from the DB. So in a sense that BlockingCollection is the database. — Henley, Dec 06 '13 at 18:47
@HenleyChiu: As long as your worker threads know when a new item is added to the database, so they aren't continually polling the database looking for new jobs, that'll work great. — Jim Mischel, Dec 06 '13 at 19:20

score 0 · Answer 2 · edited May 23 '17 at 10:26

0

The .NET thread pool isn't really designed for a fixed number of threads. It's designed to use the resources of the machine in the best way possible to perform multiple relatively small jobs.

Maybe a better solution for you would be to instantiate a fixed number of BackgroundWorkers instead? There are some reasonable BW examples.

edited May 23 '17 at 10:26

Community

1
1

answered Dec 06 '13 at 17:32

HTTP 410

17,300
12
76
127

So if I wanted 5 workers, I'd do something like: BackgroundWorker[] workers = new BackgroundWorker[5]; for(int i = 0 ; i < 5; i++) { workers[i].RunWorkerAsync() } And the DoWork method would be an infinite loop where it reads jobs from the db and processes it? – Henley Dec 06 '13 at 18:37
That's it - relatively simple. You might also want to make the BWs cancellable so that they can listen for a cancel instruction from the primary thread. – HTTP 410 Dec 07 '13 at 12:41

Thread Pool of workers in a Window service?

2 Answers2