ld linker removing an object file from a static library when creating a shared one

Question

I have a number of static libraries which I link together into one shared library. One of them, say libUsefulFunc.a contains an object file usefulFunc.o with a function, usefulFunc() that is only used from another static library, let's say usingFunc(), residing in usingFunc.c hosted in libUsingFunc.a

The problem is that the linker throws away the usefulFunc.o and I get error " undefined reference ". I tried both orders of linking.

I've recreated the situation using the simplest files I could think of:

a.h

extern int foo();

a.c

#include "a.h"
int foo()
{
    return 13;
}

b.c

#include "a.h"

extern int b()
{
  return print("a = %d\n", foo());
}

Building it all:

gcc -c a.c -o a.o
gcc -c b.c -o b.o
ar q b.a b.o
ar q a.a a.o
ld -shared -o test.so ./b.a ./a.a
nm ./test.so 
00001034 A __bss_start
00001034 A _edata
00001034 A _end

If I provide the object files instead of archives:

ld -shared -o test.so ./a.o ./b.o
nm ./test.so 
00001220 a _DYNAMIC
00000000 a _GLOBAL_OFFSET_TABLE_
00001298 A __bss_start
00001298 A _edata
00001298 A _end
000001a0 T b
00000194 T foo
         U print

Is there a way to tell the linker not to throw away the object files he thinks are unused without having to list all object files? I know there is a --whole-archive option, but I build the library as a part of Android NDK project and did not find a way to pass this option for specific library one.

An update I've fully understood my original problem and found the correct solution. First to my example above: the linker starts from the entry points and searches for all symbols they use. These are looked up in the current library. Once found it adds the symbols they use to its list and so force. The libraries are processes only once and in the order they appear on the command line. So if the second library uses a symbol from the first one - the symbol will remain undefined, as the linker does not go back. So in my example I should have told him that b() will be called externally, I could do it by using --undefined=b:

ld -shared -o test.so --undefined=b ./b.a ./a.a

In the original problem I had there was a circular reference between two static libraries. as if I had in the b archive a file b1.c with function foo_b() that is called from foo(). For such cases there are 3 possible solutions I have found:

1. List the b twice: ld -shared -o test.so --undefined=b ./b.a ./a.a ./b.a
2. Use --whole-archive
3. Use --start-group archives --end-group option. The specified archives are searched repeatedly until no new undefined references are created.

For Android NDK libraries, only the first and the second options seem to be available, as NDK's makefiles don't provide a way to specify the archive group

Hope this will be useful to other people as well!

Answer 1

try with --whole-archive option:

ld -shared -o test.so --whole-archive ./a.a ./b.a

Answer 2

OK in case it's useful for followers, I have found a very useful (and odd) behavior.

If you have this linking order on your command line:

-lsomething1 -lsomething2

if the various .o files contained within libsomething1.a "refer to each another" (have some shared methods or the like, amongst themselves) and also at least one of the .o files has a dependency that is "useful/used" by the program, then all the inter-linking .o files will be loaded "when that library is linked against" (basically, when the linker hits the -lsomething1 command).

The linker then continues on and tries to load library "something2". If there is a dependency in "something2" to a .o file in "something1"

a) if there was an inter dependency between the .o files in "something1" then it will load, even if that particular dependency was unused earlier.

b) if there was no interdependency between .o files in something1 then it is possible the .o file was, as the OP phrased it, "thrown away" at load time, so will not be available.

So basically its possible to "sometimes" have objects in something1 that satisfy link time dependencies with something2, and the linker is happy, even though they are specified in the "wrong order" (something1 then something2, in this case, the right order is reverse of that, things that satisfy dependencies come later in the link chain). Sooo confusing.

Apparently if it notices it needs/wants a .o file at load time, it loads the entire .o file, not just the symbols it knows it needs/wants.

So if you accidentally remove an inter dependency, you can suddenly run into undefined reference failures where none existed before (which you can typically counter act by putting the "-l" commands into the "correct" order). And this also means that you can "accidentally" satisfy forward dependencies, which is not the way ld typically works. Go figure. You can accidentally satisfy forward dependencies [!]

You can see more precisely what is going on (which .o files are being included or not) by adding -Wl,-verbose to your linking command line.

You can see which .o files (and their symbols) are included in your .a file (all together) by using the nm command (ex: nm libmylib.a) or if you're cross compiling its something like i686-mingw-nm libmylib.a or its like. GL!

Answer 3

Avoid linking static objects into a shared library, because a shared library should preferably contain position independent code (to avoid too much relocation at dynamic link time).

In practice, recompile every source file like foo.c with gcc -fPIC -O -Wall -c foo.c -o foo.pic.o and link them all to make your shared library, e.g. gcc -shared *.pic.o -o libshared.so (and you may link needed libraries into your .so)