0

We are using GCC and Intel OpenMP.

I know that the combination of GCC and GCC OpenMP needs a special build of OpenMP (to use pthread interfaces rather than directly use the futex system call).

Is it possible to use Thread Sanitizer and GCC + Intel OpenMP? Obviously I can't just simply recompile the Intel OpenMP library.

I've started a test and I'm getting errors like

WARNING: ThreadSanitizer: data race (pid=15551)
  Atomic read of size 1 at 0x7fffec302980 by thread T4:
    #0 pthread_mutex_lock ../../../../libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:4250 (libtsan.so.0+0x52d1a)
    #1 __kmp_resume_64 <null> (libiomp5.so+0xb7ae3)

  Previous write of size 1 at 0x7fffec302980 by thread T7:
    #0 pthread_mutex_init ../../../../libsanitizer/tsan/tsan_interceptors_posix.cpp:1227 (libtsan.so.0+0x4c343)
    #1 __kmp_suspend_64 <null> (libiomp5.so+0xb587b)

  Thread T4 (tid=15590, running) created by main thread at:
    #0 pthread_create ../../../../libsanitizer/tsan/tsan_interceptors_posix.cpp:969 (libtsan.so.0+0x5ec85)
    #1 __kmp_create_worker <null> (libiomp5.so+0xb1a34)

I kind of assume that anything with the __kmp prefix should be safe, and this error is because Thread Sanitizer does not encapsulate any __kmp functions.

Paul Floyd
  • 5,530
  • 5
  • 29
  • 43
  • If you can switch to clang: https://stackoverflow.com/questions/33004809/can-i-use-thread-sanitizer-for-openmp-programs – 273K Jul 02 '23 at 16:17
  • In short, no. It's a large project and switching compiler would be non-trivial. – Paul Floyd Jul 03 '23 at 07:57

1 Answers1

1

First, TSan by it's own does not understand OpenMP synchronization like barriers or task dependencies. We developed libarcher as part of LLVM to feed this information into TSan. If you compile a code with clang -fopenmp -fsanitize=thread, libarcher will automatically be loaded and support TSan. (verify successful loading by exporting export ARCHER_OPTIONS=verbose=true). libarcher needs OMPT support in the OpenMP runtime, which libgomp does not provide yet.

libarcher also works with newer Intel OpenMP runtimes (2020+), by explicitly loading it as an OMPT tool: export OMP_TOOL_LIBRARIES=<path-to>/libarcher.so To get libarcher you can build from source. The source code is contained in the LLVM/OpenMP project: https://github.com/llvm/llvm-project/releases/download/llvmorg-16.0.6/openmp-16.0.6.src.tar.xz

tar xf openmp-16.0.6.src.tar.xz
cd openmp-16.0.6
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DOPENMP_ENABLE_LIBOMPTARGET=False -DCMAKE_INSTALL_PREFIX=$PWD/install
make -j install

should install the code to openmp-16.0.6/build/install/, so libarcher should be in openmp-16.0.6/build/install/lib/libarcher.so

Second, for the issue of false reports from OpenMP runtime code, I suggest

export TSAN_OPTIONS='ignore_noninstrumented_modules=1'

This will disable the detection of any data race caused from non-instrumented code. So, if you link to other non-instrumented libraries (e.g. BLAS) they will also be ignored. The picture TSan gets for such libraries is limited to intercepted libc functions anyways.

Joachim
  • 71
  • 6