lli: LLVM ERROR: Cannot select: X86ISD::WrapperRIP TargetGlobalTLSAddress:i64

Question

Running the following code with clang++ -S -emit-llvm main.cpp && lli main.ll on Linux(Debian)

#include <future>

int main () {
  return std::async([]{return 1;}).get();
}

fails to run on lli due to the following error:

LLVM ERROR: Cannot select: 0xd012e0: 
     i64 = X86ISD::WrapperRIP TargetGlobalTLSAddress:i64<i8** @_ZSt15__once_callable> 0 [TF=10]

 0xd020c0: i64 = TargetGlobalTLSAddress<i8** @_ZSt15__once_callable> 0 [TF=10]
In function: _ZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEJPS1_S9_SA_EEvRSt9once_flagOT_DpOT0_

Questions:

What does it mean?

Are there any compiler-flags that fix this problem?

using -stdlib=libc++ compiles and runs successfully*; what specific features is libstdc++ using that cause this issue ?

EDIT:

The motivation behind this question is to understand the differences between libc++ and libstdc++ that leads to this specific error message (on Linux) in llvm's orcjit.

On OSX gcc has been deprecated and clang uses by default libc++. To reproduce this error on OSX you probably have to install gcc & use -stdlib=libstdc++.

Here is the llvm-ir (it's unfortunately to big to embed it here directly)

what do you mean by "compiles and runs successfully" with libc++. I get: LLVM ERROR: Program used external function '_ZTVNSt3__114__shared_countE' which could not be resolved! when running lli... using a macOS? — Tobias, Feb 05 '17 at 14:12
yes you are right, apparently I'm using a custom build which contains those symbols in the binary; however I'm primary interested in understanding why llvm's orcjit fails to run this code *with* gcc standard library. — Gaetano, Feb 05 '17 at 19:34
https://llvm.org/bugs/show_bug.cgi?id=21431 ? i know this is mcjit but maybe the same problem for orc — Tobias, Feb 05 '17 at 20:17

score 2 · Answer 1 · edited May 23 '17 at 12:24

EDIT:

The error turned out to be caused by the lack of TLS support in the JITer. This answer describes another problem concerned with linking and lli.

If you have a look at the generated IR from clang++ -std=c++11 -S -emit-llvm test.cpp, you will find that many of the symbols, e.g. _ZNSt6futureIiE3getEv, are only declared, but never defined. The linker is never called, since -S "Only run[s] preprocess and compilation steps" (clang --help).

lli only executes the IR Module and does no "implicit" linking, how is it supposed to know which libraries to link in?

There are different solutions to this, depending on why you are using lli:

compile and link the IR Module: llc main.cpp && clang++ -lpthread main.s (pthread is required s. What is the correct link options to use std::thread in GCC under linux?)
(unconfirmed) use LD_PRELOAD="x.so y.so" to force-load the libraries before running lli
JIT the module programmatically and use LoadLibraryPermanently(nullptr) (adds symbols of the program into the search space) and LoadLibraryPermanently(file, err) for additional libs (s. http://llvm.org/docs/doxygen/html/classllvm_1_1sys_1_1DynamicLibrary.html)

I can only guess as to why libc++ works for you since it fails on my machine, but presumably it's the case because it is loaded into lli already and lli calls sys::DynamicLibrary::LoadLibraryPermanently(nullptr) to add the program's symbols to its JIT search space (s. https://github.com/llvm-mirror/llvm/blob/release_40/tools/lli/OrcLazyJIT.cpp#L110).

This does not look to me like a linker issue. It is a backend issue: the code is not even generated. — Joky, Feb 07 '17 at 16:11
@Joky true, realized that only after the back & forth on the question. It's just an additional issue. I'll edit the answer. — Tobias, Feb 11 '17 at 03:47

score 1 · Answer 2 · answered Feb 07 '17 at 16:43

The LLVM-dev mailinglist pointed out:

What does it mean?

The llvm-backend in orcjit does currently not support thread-local storage(TLS)

a minimal example is:

extern thread_local int tls;
int main() {
    tls = 42;
    return 0;
}

using -stdlib=libc++ compiles and runs successfully*; what specific features is libstdc++ using that cause this issue ?

this works because libc++ future::get implementation does not use thread_local keyword.

Are there any compiler-flags that fix this problem?

currently there is no solution. Using lli -relocation-model=pic trades this problem with a relocation failure.

lli: LLVM ERROR: Cannot select: X86ISD::WrapperRIP TargetGlobalTLSAddress:i64

2 Answers2