2

Working on embedded device (ARM, uClibc), I have a static executable which statically linked with different libraries, and have dynamic loading feature using dlopen.

set(EXTERNAL_LIBS "-lpthread -lpcap -lcurl -ldl")    
target_link_libraries(myApp -static ${EXTERNAL_LIBS})

When loading simple plugin everything works fine.

void plugin::execute() {
   std::cout << "hello world" << std::endl;
}

When adding string variable:

void plugin::execute() {
    //THIS IS NOT WORKING
    std::string test = "hello world from thing";
    std::cout << test << std::endl;
}

I get:

"can't resolve symbol '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1EPKcRKS3_'"

I've tried adding -rdynamic as suggested here: dlopen a dynamic library from a static library, when the dynamic library uses symbols of the static one by adding:

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -rdynamic  -Wl,-export-dynamic")

But it still doesn't work.

The missing symbol DOES exists in the static binary (verified using nm)

What am I missing here??


Added simplified output of the build process:

Compiling object files

arm-linux-uclibcgnueabi-g++  -fPIC   -std=gnu++98  -o CMakeFiles/libstaticlib.dir/test1.cpp.o   -c /work/src/test1.cpp
arm-linux-uclibcgnueabi-gcc  -fPIC   -std=gnu++98  -o CMakeFiles/libstaticlib.dir/test2.cpp.o   -c /work/src/test2.cpp

Linking CXX static library

arm-linux-uclibcgnueabi-ar qc libstaticlib.a  CMakeFiles/libstaticlib.dir/test1.cpp.o CMakeFiles/libstaticlib.dir/test2.cpp.o
arm-linux-uclibcgnueabi-ranlib libstaticlib.a

Compiling myApp

arm-linux-uclibcgnueabi-g++   -fPIE   -std=gnu++98 -o CMakeFiles/myapp.dir/main.cpp.o -c /work/src/main.cpp

Linking CXX executable

arm-linux-uclibcgnueabi-g++   -rdynamic CMakeFiles/myapp.dir/main.cpp.o  -o myapp  -L/work/lib -Wl,-rpath,/work/lib -rdynamic -static libstaticlib.a -lpthread -lpcap -lcurl -ldl

Compiling plugin

arm-linux-uclibcgnueabi-g++  -fPIC   -std=gnu++98 -o CMakeFiles/plugin.dir/plugin/plugin.cpp.o -c /work/src/plugins/plugin/plugin.cpp

Linking CXX shared library ../libplugin.so

arm-linux-uclibcgnueabi-g++  -fPIC   -shared -Wl,-soname,libplugin.so -o ../libplugin.so CMakeFiles/plugin.dir/plugin/plugin.cpp.o  -L/work/lib

output of readelf -s myapp | grep ...:

 0021ce74    68 FUNC    WEAK   DEFAULT    2 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1EPKcRKS3_
MukiD
  • 31
  • 3
  • "The missing symbol DOES exists in the static binary (verified using nm)". Please add as evidence the actual output of `readelf --dyn-syms myapp | grep _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1EPKcRKS3_` – Mike Kinghan Nov 22 '18 at 15:54
  • readelf --dyn-syms myapp have no output (maybe because myapp is static?) nm gives the following: 0021ce74 W _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1EPKcRKS3_ – MukiD Nov 22 '18 at 16:27
  • `nm` is showing you the global symbol table. `nm -D` will show the dynamic symbol table, and it will not be there. It's not being exported for dynamic linkage, despite `-rdynamic`, but without your code or an [mcve] that's as far as I can get, sorry. – Mike Kinghan Nov 22 '18 at 18:28
  • Figured it, and learned something. See updated answer. – Mike Kinghan Nov 22 '18 at 19:11

1 Answers1

2

-rdynamic is a GCC linkage option. So you can pass it directly to GCC when it invokes the linker (ld). The effect of -rdynamic is to make GCC pass --export-dynamic in its invocation of ld, as you may see in the GCC manual: 3.14 Options for Linking

--export-dynamic is not a GCC option, but is an ld option. You can tell GCC to pass this option when it invokes ld by passing -Wl,--export-dynamic to GCC.

So your GCC options:

-rdynamic  -Wl,-export-dynamic

do the same thing twice: -rdynamic would be enough.

But the setting:

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -rdynamic")

will not cause GCC to pass -rdynamic when it invokes the linker.

That is because CMAKE_CXX_FLAGS sets the options that will be passed to each C++ compilation. Since no linkage happens in compilation, linkage options are ignored and have no effect. Linkage options should be set in CMAKE_EXE_LINKER_FLAGS, like:

set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -rdynamic")

But even then...

the title of your question will remain true, because -rdynamic doesn't work from a static binary, period.

From the linker man page

--export-dynamic

When creating a dynamically linked executable, using the -E option or the --export-dynamic option causes the linker to add all symbols to the dynamic symbol table.

My emphasis. And you are not creating a dynamically linked executable, because you are linking -static. There will be no dynamic symbol table to which all symbols could be added.

Here's an elementary demonstration.

main.c

int foo() {
    return 0;
}

int main()
{
    return foo();
}

Compile and link normally:

$ gcc main.c

Dynamic symbol table:

$ nm -D a.out 
                 w __cxa_finalize
                 w __gmon_start__
                 w _ITM_deregisterTMCloneTable
                 w _ITM_registerTMCloneTable
                 U __libc_start_main

Compile and link -rdynamic; see dynamic symbol table:

$ gcc -rdynamic main.c
$ nm -D a.out 
0000000000201010 B __bss_start
                 w __cxa_finalize
0000000000201000 D __data_start
0000000000201000 W data_start
0000000000201010 D _edata
0000000000201018 B _end
0000000000000884 T _fini
00000000000007ea T foo
                 w __gmon_start__
00000000000006a0 T _init
0000000000000890 R _IO_stdin_used
                 w _ITM_deregisterTMCloneTable
                 w _ITM_registerTMCloneTable
0000000000000880 T __libc_csu_fini
0000000000000810 T __libc_csu_init
                 U __libc_start_main
00000000000007f5 T main
00000000000006e0 T _start

More symbols now, including main and foo.

Compile and link -static; see dynamic symbol table:

$ gcc -static main.c
$ nm -D a.out 
nm: a.out: no symbols

And finally:

$ gcc -rdynamic -static main.c
$ nm -D a.out 
nm: a.out: no symbols

You just cannot link a program statically if you want a plugin to reference symbols that it defines.

Mike Kinghan
  • 55,740
  • 12
  • 153
  • 182
  • @MukiD :( You better clean and rebuild your project from scratch with `make VERBOSE=1`, then edit you post to show the complete output of that so we can see what's really going on. – Mike Kinghan Nov 22 '18 at 14:34
  • Thanks for the demonstration. you have some idea for a workaround? I'm using static binary to avoid dependencies issues over different platforms – MukiD Nov 22 '18 at 19:22
  • Is it possible to make a single shared lib of all "myApp" dependencies, including uclibc? And link myApp against it dynamically? – MukiD Nov 23 '18 at 06:17
  • @MukiD Only if you have static libraries of all of those dependencies, recursively, whose contents that have been compiled `-fPIC`, so as to be linkable into your kitchen-sink shared library. I don't think you'll find those or that it will be practical to make them yourself. – Mike Kinghan Nov 23 '18 at 07:39