Can Intel oneTBB co-exist with OpenMP?

Question

I have a quick theoretical question. Can Intel oneTBB (Thread Building Blocks) co-exist with OpenMP (Open MultiProcessing) in one code base? Both are parallel runtimes for shared memory architectures. I have a parallel code with oneTBB. I introduced OpenMP parallel for pragmas into it. The code compiles and runs, but I see no performance benefit from using OpenMP. I'm starting to think these runtimes are mutually exclusive.

Update: Please note, I don't want to choose one of the runtimes as in the question suggested (C++ Parallelization Libraries: OpenMP vs. Thread Building Blocks). I want to use both of them simultaneously. To do that I would like to find out, if it is at all possible by design or these libraries are mutually exclusive.

Does this answer your question? [C++ Parallelization Libraries: OpenMP vs. Thread Building Blocks](https://stackoverflow.com/questions/615264/c-parallelization-libraries-openmp-vs-thread-building-blocks) — Soleil, Mar 13 '23 at 15:33
Thank you for your suggestion. It is a good question with good answers, but it doesn't solve my problem. I don't want to choose one of them as in the question above. I wan't to use both of them simultaneously. To do that I would like to find out, if it is at all possible by design or these libraries are mutually exclusive. — mabalenk, Mar 13 '23 at 15:59
What does your loop body look like? Is it just making TBB function calls inside a loop, or do you have some loops that manually loop over arrays for `#pragma omp` SIMD and/or parallel vs. other code that calls TBB functions? OpenMP may only be able to parallelize when it can see definitions for all the functions to be sure they only access pointed-to data, not any globals or something which would create a data race. — Peter Cordes, Mar 13 '23 at 16:08
It is the second scenario: TBB has its own parts of the code that it parallelises and OpenMP has its own loops that it parallelises. They do not intersect or call each others' functions or use each others' variables. I would like them to co-exist and each do their own thing. Is that possible? — mabalenk, Mar 13 '23 at 16:30
I'd expect so; OpenMP will start some worker threads, but they'll be idle while your program is inside a TBB function call. (Assuming you don't manually multi-thread except for TBB and OpenMP.) — Peter Cordes, Mar 13 '23 at 16:41
Are the TBB and OpenMP phases alternating with each other or do you want OpenMP threads and TBB threads to be active at the same time? — Michael Klemm, Mar 13 '23 at 17:32
In the "duplicate" OP, several answers specify that *they are not mutually exclusive*. In which way this is not sufficient ? Please update your OP. — Soleil, Mar 13 '23 at 18:00
@Soleil, yes, the third answer by Nikhil in the ['sister' question](https://stackoverflow.com/questions/615264/c-parallelization-libraries-openmp-vs-thread-building-blocks) mentions it as a side remark without giving any details. I was hoping to get more information here. I'm sorry, but I disagree with you. In my opinion these are two different questions. The point is not 'OpenMP _vs_ TBB' it is 'OpenMP _and_ TBB'. — mabalenk, Mar 13 '23 at 22:20

score 2 · Accepted Answer · answered Mar 14 '23 at 08:37

2

If the phase between TBB and the OpenMP API are alternating, then a pattern like this might turn out to be the solution (from the OpenMP side):

void omp_parallel() {
    // do work with OpenMP directives
}

void tbb_parallel() {
    // do work with TBB templates
}

void alternating_omp_and_tbb() {
    for (int i = 1; i < 10; i++) {
        omp_parallel();
        omp_pause_resource_all(omp_pause_soft);
        tbb_parallel();
    }
}

Please have a look at omp_pause_resource and omp_pause_resource_all for what's possible from the OpenMP side of things.

Note, that if you do this, the first encounter of OpenMP directives will have overheads, as the OpenMP runtime will have to spin up threads and initialize them again.

I'm not a big TBB user anymore, but TBB also seems to have such a shutdown method. See https://oneapi-src.github.io/oneTBB/main/tbb_userguide/Initializing_and_Terminating_the_Library.html.

answered Mar 14 '23 at 08:37

Michael Klemm

2,658
1
12
15

1

Wouldn't it be better to let the OpenMP worker threads just sit idle, instead of stopping them and restarting them? Unless you're very tight on RAM, having some extra threads that sleep during your TBB phases won't be problematic for the OS. OMP threads shouldn't have a big memory footprint, should they? – Peter Cordes Mar 14 '23 at 12:17
2

Depends... The OpenMP threads may continue spin-waiting (the OpenMP spec is silent about this, except `OMP_WAITPOLICY=passive`). I also have put `omp_pause_soft`, which does not fully wind down OpenMP's state, but reduces it's consumption on the execution units. – Michael Klemm Mar 14 '23 at 15:13
1

Good point, yeah that's a possible problem, especially the OMP and TBB regions are short with frequent switches between the two; OMP workers could steal a lot of CPU time from TBB workers if they busy-wait for a while for new work. Definitely something to test on the OpenMP implementation(s) you actually compile for. – Peter Cordes Mar 14 '23 at 15:16

Can Intel oneTBB co-exist with OpenMP?

1 Answers1