6

The following code should be pretty straightforward but seems to end up in a hanging situation when trying to do a .join() on the threads with nested OpenMP code. Using GCC compiler 4.7.2 x64 with pthreads from http://sourceforge.net/projects/mingwbuilds with g++ threadexample.cpp -Wall -std=c++11 -fopenmp -o threads

// threadexample.cpp
#include <iostream>
#include <thread>
#include <omp.h>

using namespace std;

void hello(int a) {

    #pragma omp parallel for
        for (int i=0;i<5;++i) {
            #pragma omp critical
            cout << "Hello from " << a << "! " << "OMP thread iter " << i << endl;
        }

    cout << "About to return from hello function" << endl;
}

int main (int argc, char ** argv) {

    thread t1(hello, 1); //fork
    cout << "t1 away!" << endl;
    thread t2(hello, 2);
    cout << "t2 away!" << endl;

    t1.join(); //join
    cout << "thread 1 joined" << endl;
    t2.join();
    cout << "thread 2 joined" << endl;

    return 0;
}
illumi
  • 274
  • 5
  • 17
  • Have you read about what that openmp pragma does to the thread? I wonder if it already calls join for that thread id, thus causing it to hang. – Brady Nov 02 '12 at 15:00
  • Not sure. If you add `while (t1.joinable()) { cout << "t1 joinable" << endl; }` between the launch and join it seems to always be joinable. – illumi Nov 02 '12 at 15:18
  • Strange, it works for me. GCC 4.7.2 (compiled from sources) on 64-bit Scientific Linux 6.3. Although I don't think the OpenMP standard covers such cases at all, it defines nested parallelism for `paralell` regions though so you could implement it with two `section` directives instead. – Hristo Iliev Nov 02 '12 at 15:37
  • That's annoying that it works on Linux. I am using Windows mingwbuilds which I guess _may_ be causing a problem. – illumi Nov 02 '12 at 16:13
  • Well, it happens from time to time, especially when you rely on unspecified behaviour. – Hristo Iliev Nov 02 '12 at 21:59

1 Answers1

7

Mixing OpenMP and any other threading library (pthreads, Win32 threading, etc.) might not be a good idea. The OpenMP run-time might be written with the presumption that it is fully in control of threading and might not support parallel regions being run concurrently (e.g. it might use global variables like semaphores to control the thread pools).

A better pure OpenMP way to implement this would be:

#include <iostream>
#include <omp.h>

using namespace std;

void hello(int a) {

    #pragma omp parallel for
    for (int i=0;i<5;++i) {
        #pragma omp critical
        cout << "Hello from " << a << "! " << "OMP thread iter " << i << endl;
    }

    cout << "About to return from hello function" << endl;
}

int main (int argc, char ** argv) {

    omp_set_nested(1);

    #pragma omp parallel sections num_threads(2)
    {
       #pragma omp section
       {
           hello(1);
       }
       #pragma omp section
       {
           hello(2);
       }
    }

    return 0;
}

The call to omp_set_nested() is needed in order to enable nested parallelism which is disabled by default.

Hristo Iliev
  • 72,659
  • 12
  • 135
  • 186
  • Sure that is possible, but I have been using OpenMP inside QtThreads successfully. I don't see why there is bad interaction between C++11 threads and OpenMP in my case. – illumi Nov 02 '12 at 16:13
  • 5
    Both QtThreads and C++11 threads are wrappers around the system threads library, most possibly `pthreads` on most Unix systems. The OpenMP runtime of GCC is also implemented as a wrapper around `pthreads`. It might work in your case and in many others, but is still an undefined behaviour. – Hristo Iliev Nov 02 '12 at 16:25
  • 1
    @HristoIliev Is it indeed UB to use openMP together/mixed with std::thread? If so, shouldn't the compiler warn about that? Shouldn't the openMP and/or the C++11 standard be specific about that? Does this mean that openMP shouldn't be fully C++11 implementable/compatible? – Walter Dec 11 '12 at 16:10
  • @Walter, the OpenMP standard defines "*the* initial thread" as the one that executes the sequential part of the code. Note the use of a definite article. You can implement the OpenMP run-time system on top of C++11 threads but the standard does not say anything about interoperability with other threading libraries. Also the latest OpenMP standard refers to ISO/IEC 14882:1998 for the C++ part. – Hristo Iliev Dec 11 '12 at 17:08
  • @Walter, according to a colleague of mine, who is on the OpenMP language committee, all x86 implementers commit to supporting mixing other threading paradigms with OpenMP, but this is an implementation-specific behaviour rather than a standard feature (and obviously GCC doesn't do it right). – Hristo Iliev Dec 11 '12 at 17:15
  • @HristoIliev This does seem to imply that using openMP with C++ >98 is not intended/supported by the standard and implies entering murky waters. For many practical purposes it may be okay, but there is no standard that one can base one's programs on. Correct? – Walter Dec 12 '12 at 08:50
  • @Walter, that's correct. As I said, the x86 implementers would most likely provide run-times that can be used in that way, but this would not mean that it will work on other platforms (e.g. implementing support for it might be very expensive and some vendors push for certain features not to be part of the standard as they might have severe impact on the run-time performance). And as far as I understood, OpenMP 4.0, which is due early 2013, would not address other threading paradigms too. – Hristo Iliev Dec 12 '12 at 09:16