How does Intel TBB concurrent_queue work? Does it achieve fine-grained parallelism?

Question

I am new to Intel TBB. I am using concurrent_queue in order to achieve fine-grained parallelism in my project. I got few doubts. This is how i am implementing.

   thread_fun(arguments) {
      while(concurrent_queue.try_pop(v))
          operation on v;  //each thread executes its own operation(seperate v for each thread)
   }
   main() {
      for(..)
         concurrent_queue.push(); //fill the queue
      _beginthreadex(..); //create 8 win32 threads and pass concurrent_queue as an argument
   }

1. I am explicitly mentioning the number of threads. I read that TBB will create threads based on the processor core count. How can i achieve that? So that i don't need to create threads explicitly with _beginthreadex function?

2. Am i achieving fine-grained parallelism by using the concurrent_queue?

3. What do you mean by task level parallelism? How do you achieve task level parallelism with intel tbb? I am popping element from the queue. Does pop operation considered as a task? That means, all pop operations are considered as different tasks. I am popping 8 elements at a time with 8 threads. That means, i am achieving task level parallelism. Am i correct?

4. If i increase the number of threads to 32 on a quad-core processor(support 8 threads), how does the concurrent_queue work? Does only 8 threads concurrently executed on the queue or total 32 threads are executed concurrently?

Please help me.

Answer 1

TBB has a lot of parts. Concurrent_queue is just a queue that can be safely accessed by multiple threads. Concurrent queue is not directly part of the "fine grain parallelism" or "task parallelism" part of TBB. (Although those parts can use concurrent_queue.)

1. You create fine grained/task parallelism in tbb by using tbb::parallel_for. See the examples on that page or at this stack overflow question. When you use parallel_for tbb will take care of all the thread creation and will also do dynamic load balancing.

2. Concurrent_queue doesn't create any parallelism at all: it just allows multiple threads to safely (and relatively efficiently) access the same queue. It ensures there are no race conditions. In your example you are getting no more than 8 threads worth of concurrency. Actually: concurrent_queue is serializing accesses to the queue. Your parallelism is coming from your "operations on v". The accesses to the concurrent_queue should be thought of as overhead.

3. You are implementing a basic kind of task level parallelism in your example. I found this brilliant description of the difference between a task and a thread from a Stack Overflow answer by @Mitch Wheat:

   A task is something you want doing.

   A thread is one of possibly many workers who perform that task.

   More technically: tasks must not block waiting for one another. Your tasks are the "operations on v" that you are performing. You are achieving task-level parallelism. The advantage of using something like tbb::parallel_for is that it (a) automatically starts the threads for you, (b) tries to automatically choose a number of threads most appropriate for the number of cores on your user's system, (c) is optimized to reduce the overhead of distributing tasks to threads, and (d) also does some pretty cool dynamic load balancing if your task operations aren't of equal length.

4. Let's distinguish between concurrency, and parallelism. Your machine has 8 hardware threads. So you have a maximum cpu parallelism of 8. I would use the term concurrency to mean something like "the maximum parallelism you could achieve if you had a magical processor with an infinite number of cores." Typically the main reason for creating more threads than you have cores is if your threads are doing a lot of long-latency i/o operations. The operating system can switch to one of the extra threads to do cpu work while the other thread is waiting for the disk/network.