3

I'm trying to build my executable (that depends on library utils.so) using the following command

g++ -L/path/to/libutils -lutils -I/path/to/utils_headers executable.cpp -o executable

Actually I don't have utils.so - only the header files of utils library.

I'm getting the error:

ld: cannot find -lutils

Does linker really need to access all the libraries my executable depends on in order to build my executable? If it does then I'd like to know why it needs to access them.

My executable is a shared library. I'm sure that header files of the utils lib are enough to build it (i.e without having utils.so).

Alexey
  • 710
  • 3
  • 7
  • 19
  • Yes it does. How can an executable be executed if there are missing symbols? – Matthieu Brucher Dec 15 '18 at 11:02
  • 1
    I'm not running it - I'm just building it. There may be unresolved symbols, right? – Alexey Dec 15 '18 at 11:03
  • Yes. If you want to build an executable, you need to have all the symbols. – Matthieu Brucher Dec 15 '18 at 11:04
  • 1
    What do you mean by "have all the symbols"? I have all the symbols - they are declared in header files. That is enough to compile the source files and put the object files into an executable. – Alexey Dec 15 '18 at 11:06
  • 1
    In that case why do you add `-lutils`? If all the symbols are in the header, then no need to link against the library. But if they are not, then you need to link against it. – Matthieu Brucher Dec 15 '18 at 11:06
  • I didn't add it. This is the makefile I got. I can remove -lutils from it. But if I remove -lutils then how my executable know that it depends on utils.so? I suppose the linker just need -lutils to put this name into my executable - but it need not access utils.so. – Alexey Dec 15 '18 at 11:08
  • @Alexey So is `utils` a header only library maybe? – πάντα ῥεῖ Dec 15 '18 at 11:15
  • 2
    @πάνταῥεῖ I think the point is that `utils` is a shared library, only needed when an executable that needs it runs. Strictly speaking only the information in the header files would be needed to compile, so this behaviour _looks_ like an extra safety check. – juanchopanza Dec 15 '18 at 11:26
  • @πάνταῥεῖ No, it's not only header lib. What does the option -utils causes the linker to do? Does it really access this library for some reason? – Alexey Dec 15 '18 at 11:28
  • @juanchopanza You are right that utils.so is only needed whem my executable is running. But how I can build it if the linker complains about not being able to find utils.so? What command or option should I use? – Alexey Dec 15 '18 at 11:31
  • 1
    @Alexey _"What does the option -utils causes the linker to do? Does it really access this library for some reason?"_ Besides the option should be `-lutils` it would try to lookup a library named `libutils.a` in the paths you specified using the `-L` options. even runtime linked libraries need to be linked as stubs. – πάντα ῥεῖ Dec 15 '18 at 11:31
  • @Alexey There are ways to load libraries after start-up (using `dlopen`, `dlclose` etc.) which allow you to avoid linking at compile time. It is quite an extensive topic! – juanchopanza Dec 15 '18 at 11:33
  • @πάνταῥεῖ But why the linker searches for the libraries? Are they required to generate some code for my executable? – Alexey Dec 15 '18 at 11:36
  • 1
    @Alexey The stub libraries are providing generated code to link your shared libs at runtime, that's why you need this. – πάντα ῥεῖ Dec 15 '18 at 11:38
  • @juanchopanza No, I don't want to load them after start-up. I just want to build my executable having only the header files of the utils library. All the symbols that my executable needs are declared in these header files. Is it possible? – Alexey Dec 15 '18 at 11:39
  • @πάνταῥεῖ OK. Do you mean that ld need the util.so to generate some stub lib? – Alexey Dec 15 '18 at 11:43
  • I think that the main reason that ld needs libraries that it actually checks whether all symbols are found (and to decide, that static or dynamic library will be used). In the general case, there is no need of the library to do the linking. – geza Dec 15 '18 at 20:20
  • In general, you should list libraries after the object files, not before them (or the source files that will be compiled to object code). – Jonathan Leffler Dec 15 '18 at 21:31

2 Answers2

3

The linkage option -lutils by default directs the linker to search, first in the specified library search directories (-Ldir) and then in its default search directories, for either of the files libutils.so ( shared library) or libutils.a (static library), preferring libutils.so if both of them are found in the same search directory.

If such a file is found, the linker stops searching and adds that file to the input files of the linkage, whether or not it resolves any references in the linkage. The linker cannot know whether the file resolves any references if it does not input the file.

If no such file is found, the linker gives the error: cannot find -lutils. Because you told it to find libutils.{so|a} and it could not.

You say:

My executable is a shared library

But it isn't. Your compile-and-link command:

$ g++ -L/path/to/libutils -lutils -I/path/to/utils_headers executable.cpp -o executable

is not an attempt to link a shared library. It is an attempt to link a program.1

This would be an attempt to link a shared library:

$ g++ -shared -I/path/to/utils_headers -o libexecutable.so executable.cpp -L/path/to/libutils -lutils

You cannot link a program with unresolved references. But you can link a shared library with unresolved references.

So, you could link a libexecutable.so like that, or you could link it simply like:

$ g++ -shared -I/path/to/utils_headers -o libexecutable.so executable.cpp

These are two different linkages: if they succeed they produce different output files.

In the first linkage, some symbols will (let's assume) be resolved to definitions provided in libutils.so or libutils.a (whichever one is found), and this will be reflected by:

  • libutils.so is found: The .dynamic section of libexecutable.so contains a DT_NEEDED structure that expresses a runtime dependency on libutils.so. libutils.so will need to be included in any linkage that includes libexecutable.so, but the output file of such a linkage will itself contain a runtime dependency only on libexecutable.so.

  • libutils.a is found: libexecutable.so itself contains the definitions for all the symbols it uses that are defined by object files in libutils.a.2 libexecutable.so may be included in subsequent linkages with no need for libutils.{so|a}.

In the second linkage, the .dynamic section of libexecutable.so will not express a runtime dependency on libutils.so nor will the file contain definitions of any symbols provided by libutils.{so|a}. libutils.so will (again) need to be included in an subsequent linkage that includes libexecutable.so, but the output file of such a linkage will acquire independent runtime dependencies on both libexecutable.so and libutils.so.

But, if you specify -lutils in the linkage - or any linkage - and the linker cannot find libutils.{so|a} in any of its search directories, then you get the error you observe, because you told the linker to input a file, whose effects on the linkage can only be determined and implemented if that file is found - and it cannot be found.


[1] An attempt that is likely to fail, because it consumes libraries before the object files that refer to them

[2] See static-libraries to understand why.

Mike Kinghan
  • 55,740
  • 12
  • 153
  • 182
2

In general, an ELF linker needs a sufficiently accurate representation of the shared object that is linked in. It does not have to be an actually working shared objects, just a sufficiently close representation of it. A few things absolute require data that is not available in the object itself:

  • When compiling C programs, a reference to a global data object of incomplete type does not contain size information. The linker cannot place the object into the data segment unless it obtains the size information from somewhere. By default (when compiling for executables, including PIE) the object needs to be allocated in the data segment on many targets because of the relocations the compiler uses for compiling accesses to global data objects.
  • Similarly, the link editor might get the alignment of global data objects wrong if it has insufficient information.
  • Many libraries use symbol versioning. Symbol version information is only available when the link editor can see the shared object. If that information is missing, the link editor will not emit a symbol version, which instructs the dynamic linker to bind the symbol to the base version at run time, leading to subtle bugs.

However, if you only use C function symbols (not data symbols, or the varieties of symbols that C++ requires) and the target library does not use symbol versioning, you can use a stub library for linking. This is a library that defines all the functions you need and has the appropriate soname, but the functions are just dummies which do not actually do anything.

Florian Weimer
  • 32,022
  • 3
  • 48
  • 92