GOMP without GLIBC

Question

I would like to use musl with OpenMP. However, when I compile a simple OpenMP program such as

int main() {
    int i;
    #pragma omp parallel for
    for(i=0; i<100; i++) {
        printf("asdf\n");
    }
}

with

musl-gcc -fopenmp foo.c

I get the error

/usr/bin/ld: h_errno: TLS definition in //lib/x86_64-linux-gnu/libc.so.6 section .tbss mismatches non-TLS definition in /usr/local/musl/lib/libc.so section .bss
//lib/x86_64-linux-gnu/libc.so.6: error adding symbols: Bad value
collect2: error: ld returned 1 exit status

Apperently there is a problem related to thread local storage (TLS). How can I get musl working with OpenMP?

I'm currently using Ubuntu 14.04, GCC 4.9.1, EGLIBC 2.19, on a Intel(R) Xeon(R) CPU E5-1620.

Here is what is sent to the linker which I found from the collect2 tag from musl-gcc -v -fopenmp omp.o . I converted most of the spaces to newlines. The only difference between this and doing this without the -fopenmp option is the addition of the pthread and gomp.

-plugin /usr/lib/gcc/x86_64-linux-gnu/4.9/liblto_plugin.so
-plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/4.9/lto-wrapper
-plugin-opt=-fresolution=/tmp/ccPvNBKp.res
-plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc.a
-plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc_eh.a
-plugin-opt=-pass-through=-lpthread
-plugin-opt=-pass-through=-lc
-plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc.a
-plugin-opt=-pass-through=/usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc_eh.a
-dynamic-linker
/lib/ld-musl-x86_64.so.1
-nostdlib
-z relro
/usr/local/musl/lib/crt1.o
/usr/local/musl/lib/crti.o
/usr/lib/gcc/x86_64-linux-gnu/4.9/crtbegin.o
-L/usr/local/musl/lib
-L /usr/lib/gcc/x86_64-linux-gnu/4.9/.
omp.o
-lgomp
/usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc.a
/usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc_eh.a
-lpthread
-lc
/usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc.a
/usr/lib/gcc/x86_64-linux-gnu/4.9/libgcc_eh.a
/usr/lib/gcc/x86_64-linux-gnu/4.9/crtend.o
/usr/local/musl/lib/crtn.o

Here is the output from musl-gcc -fopenmpomp.c -Wl,--trace

/usr/bin/ld: mode elf_x86_64
/usr/local/musl/lib/crt1.o
/usr/local/musl/lib/crti.o
/usr/lib/gcc/x86_64-linux-gnu/4.9/crtbegin.o
-lc (/usr/local/musl/lib/libc.so)
/usr/lib/gcc/x86_64-linux-gnu/4.9/crtend.o
/usr/local/musl/lib/crtn.o
/usr/local/musl/lib/crt1.o

And who voted to close this as "unclear what I'm asking"? How could I be more clear? — Z boson, Oct 13 '14 at 12:39
There still seems to be a missing premise here; you are cross-compiling for some embedded system? — bmargulies, Oct 13 '14 at 12:49
@bmargulies no, I only added the embedded tag because these alternatives to libc are often used on embedded systems so I thought that it might attract people with experience getting OpenMP to work on these alternative libc libraries. I'll remove the tag. — Z boson, Oct 13 '14 at 12:54
glibc does not link statically on Ubuntu unless you go to great lengths. So I'm back to puzzled. — bmargulies, Oct 13 '14 at 13:59
@bmargulies, you're preaching to the choir! That's exactly why I asked this question. I don't want to link statically to `glibc` I want to link statically to `musl`. Didn't I make that clear in my question? How could I make it more clear? The title even says "GOMP without GLIBC". I'm aware glibc has issues linking statically (though I have had less problems than others) http://stackoverflow.com/questions/26304531/compiling-with-static-libgcc-static-libstdc-still-results-in-dynamic-depende/ — Z boson, Oct 13 '14 at 14:55
@bmargulies, I don't like what's happened to my question. I want to use OpenMP with _musl_ and it's not working for me. How hard is that to understand? I can't get it working using dynamic or static linkage. The reason I want to do this is irrelevant and just a red herring. Do you have experience with GOMP? Do you know how to get it to working with _musl_? — Z boson, Oct 13 '14 at 17:58
@bmargulies, BTW, is there a tag for theses alternatives to libc suh as _musl_. I added the embedded tag because that's the closest tag I could think of to cover _musl_. But that just caused confusion. What tag can I use that won't cause confusion but will still attract the expertise of those that have experience with libraries such as _musl_? — Z boson, Oct 13 '14 at 18:17
Linking dynamically with a different C library is a tricky job as you also has to specify the library-specific ELF interpreter, a.k.a. the dynamic linker - see [here](http://stackoverflow.com/questions/847179/multiple-glibc-libraries-on-a-single-host). I guess `musl-gcc` should do that for you, but if there is an option to enable some kind of verbose mode and see what commands are actually executed, paste them here for further problem analysis. — Hristo Iliev, Oct 13 '14 at 22:25
@HristoIliev, yes, `musl-gcc` takes care of this. I can compile and run fine without the `-fopenmp` option. I added what is sent to the linker to my question. — Z boson, Oct 14 '14 at 07:31
It is probably picking the wrong `libpthread.so`, though this should not happen given the `-nostdlib`. Try adding `-Wl,--trace` to get the list of object/library files that are being processed by the linker. — Hristo Iliev, Oct 14 '14 at 08:53
@HristoIliev, I added the result of `-Wl,--trace` to my question. I think `musl` implements its own pthreads. There is no `libpthread.so` in `/usr/local/musl/lib/`. However, it does contain `libpthread.a`. If I try compiling statically I get other errors (e.g. `undefined reference to `__sprintf_chk'`). — Z boson, Oct 14 '14 at 09:03
`musl` is a monolithic library that puts everything in `libc.(a|so)` and all other libraries are simply empty archives. Though not directly the cause of your particular problem, but you cannot build against `musl` OpenMP programs using GCC that was built against GLIBC for the simple reason that `libgomp` uses the non-standard `pthread_attribute_setaffinity_np()` which is not present in `musl`. You should probably rebuild `libgomp` with the `posix` configuration, loosing any Linux optimisations and the ability to bind threads. — Hristo Iliev, Oct 14 '14 at 14:18
@HristoIliev, thank you, that sounds like what I found in a link I originally added to my question but removed because it was on android and people it confused some people. I can take care are binding the threads myself but what kind of Linux optimizations will I loose? — Z boson, Oct 14 '14 at 15:40
@HristoIliev, hmmm...I use `pthread_attribute_setaffinity_np` to bind the threads myself so actually I can't bind them the way I'm doing it now either. — Z boson, Oct 15 '14 at 08:50
Take a look at the source code of `libgomp`. There is a directory called `config` with various implementations of platform-specific things. There is a `linux` directory and a `posix` directory with the latter containing implementations that use POSIX calls only. I think one thing you probably lose is the quick barrier implementation. As for the affinity setting, you can always use the Linux-specific `sched_setaffinity(2)` - it is present in `musl`. — Hristo Iliev, Oct 15 '14 at 12:17

score 0 · Answer 1 · answered Oct 12 '17 at 22:59

October 2017

Some similar experience, perhaps related, perhaps unrelated.

I got the same sort of error:

/usr/bin/ld: h_errno: TLS definition in //lib/i386-linux-gnu/libc.so.6 section .tbss mismatches non-TLS definition in /usr/lib/i386-linux-musl/libc.so section .bss
//lib/i386-linux-gnu/libc.so.6: error adding symbols: Bad value
collect2: error: ld returned 1 exit status

When trying to link libz.so using musl-gcc. My solution was to get the zlib source code and compile it using musl-gcc. There may be some additional linking issues for you, depending on the dependencies of openmp.

For me, static compilation of libz.a works fine. I still need to iron out some linking issues to compile a libz.so, but I really don't need to - the musl-gcc static executable is 8 times smaller than the gcc static executable, and that is good enough.

GOMP without GLIBC

1 Answers1