3

I am programming using pthreads in C.

I have a parent thread which needs to create 4 child threads with id 0, 1, 2, 3. When the parent thread gets data, it will set split the data and assign it to 4 seperate context variables - one for each sub-thread. The sub-threads have to process this data and in the mean time the parent thread should wait on these threads. Once these sub-threads have done executing, they will set the output in their corresponding context variables and wait(for reuse). Once the parent thread knows that all these sub-threads have completed this round, it computes the global output and prints it out. Now it waits for new data(the sub-threads are not killed yet, they are just waiting).

If the parent thread gets more data the above process is repeated - albeit with the already created 4 threads.

If the parent thread receives a kill command (assume a specific kind of data), it indicates to all the sub-threads and they terminate themselves. Now the parent thread can terminate.

I am a Masters research student and I am encountering the need for the above scenario. I know that this can be done using pthread_cond_wait, pthread_Cond_signal. I have written the code but it is just running indefinitely and I cannot figure out why.

My guess is that, the way I have coded it, I have over-complicated the scenario. It will be very helpful to know how this can be implemented. If there is a need, I can post a simplified version of my code to show what I am trying to do(even though I think that my approach is flawed!)...

Can you please give me any insights into how this scenario can be implemented using pthreads?

The Flying Dutchman
  • 582
  • 2
  • 7
  • 18
  • You may want to take a look at [this post](http://stackoverflow.com/q/9275262/777186), the answers to which describe various master/worker scenarios. There is also one answer that describes in much detail how to use POSIX for it. – jogojapan Apr 20 '12 at 05:04
  • For the synchronsation needed after all four threads did their work you might have a look here: http://www.lambdacs.com/cpt/FAQ.html#Q292 – alk Apr 20 '12 at 06:45

2 Answers2

1

As far what can be seen from your description, there seems to be nothing wrong with the principle.

What you are trying to implement is a worker pool, I guess, there should be a lot of implementations out there. If the work that your threads are doing is a substantial computation (say at least a CPU second or so) such a scheme is a complete overkill. Mondern implementations of POSIX threads are efficient enough that they support the creation of a lot of threads, really a lot, and the overhead is not prohibitive.

The only thing that would be important if you have your workers communicate through shared variables, mutexes etc (and not via the return value of the thread) is that you start your threads detached, by using the attribute parameter to pthread_create.

Once you have such an implementation for your task, measure. Only then, if your profiler tells you that you spend a substantial amount of time in the pthread routines, start thinking of implementing (or using) a worker pool to recycle your threads.

Jens Gustedt
  • 76,821
  • 6
  • 102
  • 177
  • My program will create 160000 threads when it uses this pooling concept. If I dont use thread pools, then this number goes up to 1.6 million - a 10 time increase. Would it still not affect the times?? – The Flying Dutchman Apr 20 '12 at 17:37
  • @TheFlyingDutchman, how shall I know. You didn't give us any hint of what your threads do for real, how much time they spend for the applicative task and how much is overhead. Measure and you will know. (And please share.) – Jens Gustedt Apr 20 '12 at 18:33
  • @TheFlyingDutchman - if you create only 16 work threads in your pool, and never make any more, then there will only ever be 16 work threads. – Martin James Apr 20 '12 at 21:28
0

One producer-consumer thread with 4 threads hanging off it. The thread that wants to queue the four tasks assembles the four context structs containing, as well as all the other data stuff, a function pointer to an 'OnComplete' func. Then it submits all four contexts to the queue, atomically incrementing a a taskCount up to 4 as it does so, and waits on an event/condvar/semaphore.

The four threads get a context from the P-C queue and work away.

When done, the threads call the 'OnComplete' function pointer.

In OnComplete, the threads atomically count down taskCount. If a thread decrements it to zero, is signals the the event/condvar/semaphore and the originating thread runs on, knowing that all the tasks are done.

It's not that difficult to arrange it so that the assembly of the contexts and the synchro waiting is done in a task as well, so allowing the pool to process multiple 'ForkAndWait' operations at once for multiple requesting threads.

I have to add that operations like this are a huge pile easier in an OO language. The latest Java, for example, has a 'ForkAndWait' threadpool class that should do exactly this kind of stuff, but C++, (or even C#, if you're into serfdom), is better than plain C.

Martin James
  • 24,453
  • 3
  • 36
  • 60