8

When I generate LLVM IR Code from C++, I can use the console command clang++ -emit-llvm –S test.cpp to get a test.ll file which is the LLVM IR I want.

To get an executable these are the steps to follow:

  • llvm-as test.ll -> gives me the test.bc file.

  • llc test.bc --o test.s -> gives me the test.s file.

  • clang++ test.s -o test.native -> gives me a native file that i can execute.

For C++ this works just fine.

In theory, should the same steps apply when I write Rust or Python Code?

I take my Rust code and get the LLVM IR by typing rustc test.rs --emit llvm-ir. This gives me the test.ll file again.

For Python, I use "Numba" and get the LLVM IR by typing numba --dump-llvm test.py> test.llwhich also gives me the test.ll file.

The steps to generate an executable from those .ll files should be the same.

They work up until the last step that creates the native executable:

Python Error 

/tmp/test-9aa440.o: In function 'main':
test.bc:(.text+0x67): undefined reference to 'numba_gil_ensure'
test.bc:(.text+0x79): undefined reference to 'numba_unpickle'
test.bc:(.text+0x84): undefined reference to 'PyObject_Str'
test.bc:(.text+0x8f): undefined reference to 'PyString_AsString'
test.bc:(.text+0xa1): undefined reference to 'PySys_WriteStdout'
test.bc:(.text+0xa9): undefined reference to 'Py_DecRef'
test.bc:(.text+0xb1): undefined reference to 'Py_DecRef'
test.bc:(.text+0xbd): undefined reference to 'PySys_WriteStdout'
test.bc:(.text+0xc5): undefined reference to 'numba_gil_release'
test.bc:(.text+0xff): undefined reference to 'numba_gil_ensure'
test.bc:(.text+0x10b): undefined reference to 'PySys_WriteStdout'
test.bc:(.text+0x113): undefined reference to 'numba_gil_release'
clang: error: linker command failed with exit code 1 (use -v to see     invocation)

Rust Error 

/tmp/main-5e59bd.o: In function ‘main::sum::h514304ffa40dd7c3’:
main.bc:(.text+0xf): undefined reference to ‘core::panicking::panic::h2596388ccef1871c’
/tmp/main-5e59bd.o: In function ‘main’: main.bc:(.text+0x53): undefined reference to ‘std::rt::lang_start::h65647f6e36cffdae’
clang: error: linker command failed with exit code 1 (use -v to see invocation)

What I get from this is that clang doesn't understand the Rust/Python specific parts of the LLVM IR file (e.g. "PyObject" in Python or "panic" from Rust) that were used to generate the .bc, .s and in theory the .native files.

But why are those even in the IR in the first place? Shouldn't the LLVM IR be uniform and those parts be transformed so the LLVM toolchain can work with them? As far as I know LLVMs modularity should allow those steps by using LLVM IR. Is there maybe another way to do this i don't know about?

Can I generate the IRs from those languages in some other way that gives "pure" LLVM IR that clang understands, or could I still generate executables from those files, but in some other way without clang?

Machavity
  • 30,841
  • 27
  • 92
  • 100
Philipp
  • 745
  • 2
  • 7
  • 20
  • Possiby linked to: http://stackoverflow.com/questions/15169015/call-python-code-from-llvm-jit?rq=1 – MKesper Mar 06 '17 at 13:13
  • If you inspect the .ll source you should find references to externally defined functions in both Python and Rust cases. You need to link these in. The same might be true for the IR generated from C++ source, but `clang++` includes the C++ standard lib by default. –  Mar 06 '17 at 23:52

1 Answers1

6

I can speak of Rust code:

You need to link Rust's std library something like this:

$(LLI) -load /Users/Stanislaw/.rustup/toolchains/stable-x86_64-apple-darwin/lib/libstd-f5a209a9.dylib ./target/debug/jitrust.bc

See the full example of Makefile I use here.

P.S. I would assume that the same goes about Python. You have to also supply libraries which contain this "unreferenced" stuff.

Stanislav Pankevich
  • 11,044
  • 8
  • 69
  • 129
  • 1
    Thanks, I used `clang PATH/TO/RUSTLIB test.s -o test.native` and could generate and run it afterwards. Python should work like this, too. I will update when i find the solution for that as well. – Philipp Mar 09 '17 at 12:53
  • 1
    I was trying to get it to work with Python in a similar fashion, but i cant do it. As i generate the IR with numba, linking only the python lib isnt enough for clang. Linking numba though is not possible as there is no .so file for that. I could find the numba functions that clang is missing but they are only available localy. Also numba as an internal python script needs to be called in a python environment. I dont see a way to compile python IR with clang, by using numba to generate the IR. It might be possible, by generateing the IR without numba, but i dont think its possible with numba. – Philipp Apr 20 '17 at 13:45