Huge memory leak when combining OpenMP, Intel MKL and MSVC compiler

Question

I am working on a considerably large C++ project with a high emphasis on performance. It therefore relies on the Intel MKL library and on OpenMP. I recently observed a considerable memory leak that I could narrow down to the following minimal example:

#include <atomic>
#include <iostream>
#include <thread>

class Foo {
public:
  Foo() : calculate(false) {}

  // Start the thread
  void start() {
    if (calculate) return;
    calculate = true;
    thread = std::thread(&Foo::loop, this);
  }

  // Stop the thread
  void stop() {
    if (!calculate) return;

    calculate = false;
    if (thread.joinable())
      thread.join();
  }

private:
  // function containing the loop that is continually executed
  void loop() {
    while (calculate) {
      #pragma omp parallel
      {
      }
    }
  }

  std::atomic<bool> calculate;
  std::thread thread;
};


int main() {
  Foo foo;
  foo.start();
  foo.stop();
  foo.start();

  // Let the program run until the user inputs something
  int a;
  std::cin >> a;

  foo.stop();
  return 0;
}

When compiled with Visual Studio 2013 and executed, this code leaks up to 200 MB memory per second (!).

By modifying the above code only a little, the leak totally disappears. For instance:

• If the program is not linked against the MKL library (which is obviously not needed here), there is no leak.
• If I tell OpenMP to use only one thread, (i.e. I set the environment variable OMP_NUM_THREADS to 1), there is no leak.
• If I comment out the line #pragma omp parallel, there is no leak.
• If I don't stop the thread and start it again with foo.stop() and foo.start(), there is no leak.

Am I doing something wrong here or am I missing something ?

Answer 1

MKL's parallel (default) driver is built against Intel's OpenMP runtime. MSVC compiles OpenMP applications against its own runtime that is built around the Win32 ThreadPool API. Both most likely don't play nice. It is only safe to use the parallel MKL driver with OpenMP code built using Intel C/C++/Fortran compilers.

It should be fine if you link your OpenMP code with the serial driver of MKL. That way, you may call MKL from multiple threads at the same time and get concurrent serial instances of MKL. Whether n concurrent serial MKL calls are slower than, comparable to or faster than a single threaded MKL call on n threads is likely dependent on the kind of computation and the hardware.

Note that Microsoft no longer support their own OpenMP runtime. MSVC's OpenMP support is stuck at version 2.0, which is more than a decade older than the current specification. There are probably bugs in the runtime (and there are bugs in the compiler's OpenMP support itself) and those are not likely to get fixed. They don't want you to use OpenMP and would like you to favour their own Parallel Patterns Library instead. But PPL is not portable to other platforms (e.g. Linux), therefore you should really be using Intel Treading Building Blocks (TBB). If you want quality OpenMP support under Windows, use the Intel compiler or some of the GCC ports. (I don't work for Intel)