GCC linker ignores symbols in a .a library that I have confirmed are present

Question

I have a really perplexing problem in GCC.

I get the following error:

gcc -Wall -Werror -L/Users/red_angel/chorebox_sys/lib -o products/chbc2c -lchorebox ofiles/main.o
Undefined symbols for architecture x86_64:
  "_chbclib_flushout", referenced from:
      _main in main.o
  "_chorebox_argc", referenced from:
      _chorebox_command_line in libchorebox.a(chorebox_command_line.o)
  "_chorebox_argv", referenced from:
      _chorebox_command_line in libchorebox.a(chorebox_command_line.o)
  "_chorebox_env", referenced from:
      _chorebox_command_line in libchorebox.a(chorebox_command_line.o)
  "_mn_command_options", referenced from:
      _main in main.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [products/chbc2c] Error 1

What's wrong with this error? I have confirmed that the _chorebox_argc symbol is indeed present in "libchorebox.a".

I confirmed it by running the command:

nm /Users/red_angel/chorebox_sys/lib/libchorebox.a | cat -n | chodo -_chorebox_argc flip

As the "chodo" command is an command I wrote that you may not be familiar - I will explain what it does. It reads from Standard Input, and forward to Standard Output every line that matches the search pattern. In this case (to make a long story short) it outputs every line containing the "_chorebox_argc" string.

I get the following output:

     3  0000000000000004 C _chorebox_argc
    55                   U _chorebox_argc

To get a closer look at the relevant part of the file, I type the same command, only this time omitting the "chodo" command at the end of the piped series of commands --- and hereby will copy/paste to you the relevant part of that file:

     1  
     2  /Users/red_angel/chorebox_sys/lib/libchorebox.a(vars.o):
     3  0000000000000004 C _chorebox_argc
     4  0000000000000008 C _chorebox_argv
     5  0000000000000008 C _chorebox_env
     6  
     7  /Users/red_angel/chorebox_sys/lib/libchorebox.a(chorebox_mlc.o):
     8  00000000000000c8 s EH_frame0
     9  0000000000000075 s L_.str
    10                   U ___stderrp
    11                   U _chorebox_argv
    12  0000000000000000 T _chorebox_mlc
    13  00000000000000e0 S _chorebox_mlc.eh
    14                   U _exit
    15                   U _fflush
    16                   U _fprintf
    17                   U _malloc
    18  
    19  /Users/red_angel/chorebox_sys/lib/libchorebox.a(chorebox_apend_string.o):
    20  0000000000000078 s EH_frame0
    21  0000000000000000 T _chorebox_apend_string
    22  0000000000000090 S _chorebox_apend_string.eh
    23                   U _chorebox_join_string
    24                   U _free
    25  

Needless to say ---- the symbol is definitely present in the "libchorebox.a" file ----- so why is the GCC linker complaining that it is not found?

Answer 1

Put the -l option after the file that needs it (ofiles/main.o)

See this question for more information on link order.

Answer 2

After some discussion in chat, we discovered that the problem lay in 'common' definitions. A simplified version of the code causing trouble follows. The system is Mac OS X (Mavericks and Yosemite).

With common definitions only

vars.h

extern int    chorebox_argc;
extern char **chorebox_argv;
extern char **chorebox_envp;

vars.c

#include "vars.h"
int    chorebox_argc;
char **chorebox_argv;
char **chorebox_envp;

main.c

#include "vars.h"

int main(int argc, char **argv, char **envp)
{
    chorebox_argc = argc;
    chorebox_argv = argv;
    chorebox_envp = envp;
    return argc;
}

Compilation 1

$ gcc -c vars.c
$ nm vars.o
0000000000000004 C _chorebox_argc
0000000000000008 C _chorebox_argv
0000000000000008 C _chorebox_envp
$ ar rv libvars.a vars.o
ar: creating archive libvars.a
a - vars.o
$ ranlib libvars.a
warning: ranlib: warning for library: libvars.a the table of contents is
empty (no object file members in the library define global symbols)
$ gcc -c main.c
$ gcc -o program main.o vars.o
$ gcc -o program main.o -L. -lvars
Undefined symbols for architecture x86_64:
  "_chorebox_argc", referenced from:
      _main in main.o
  "_chorebox_argv", referenced from:
      _main in main.o
  "_chorebox_envp", referenced from:
      _main in main.o
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status
$

Note the C in the output from nm. That indicates a 'common' definition of the variables. It isn't enough, on its own, to make them into variable definitions — witness the message from ranlib.

I'm not sure if this is new behaviour in Mac OS X or not. However, it seems that having a source file that only defines uninitialized variables isn't sufficient when the variables are defined in a library, though it is sufficient when the object file is linked directly.

With variable definitions

vardefs.c

#include "vars.h"
int    chorebox_argc = 0;
char **chorebox_argv = 0;
char **chorebox_envp = 0;

This has explicitly initialized versions of the variables. The initializer values are the same as the default values, but the explicit initialization makes all the difference.

Compilation 2

$ rm libvars.a
$ gcc -c vardefs.c
$ ar rv libvars.a vardefs.o
ar: creating archive libvars.a
a - vardefs.o
$ gcc -o program main.o -L. -lvars
$ 

The explicitly initialized variables are picked up from the library without problem.

With one variable definition

vars.h

extern int    chorebox_argc;
extern char **chorebox_argv;
extern char **chorebox_envp;
extern int make_believe;

vars.c

#include "vars.h"
int    chorebox_argc;
char **chorebox_argv;
char **chorebox_envp;

int make_believe = 59;

main.c

#include "vars.h"

int main(int argc, char **argv, char **envp)
{
    chorebox_argc = argc;
    chorebox_argv = argv;
    chorebox_envp = envp;
    make_believe  = 1;
    return argc;
}

Compilation 3

$ gcc -c vars.c
$ ar rv libvars.a vars.o
ar: creating archive libvars.a
a - vars.o
$ nm vars.o
0000000000000004 C _chorebox_argc
0000000000000008 C _chorebox_argv
0000000000000008 C _chorebox_envp
0000000000000000 D _make_believe
$ gcc -c main.c
$ gcc -o program main.o -L. -lvars
$ 

Note that adding the initialized make_believe is sufficient to pull the object file from the library, and the common definitions for the other variables are then sufficient to satisfy the linker.

Lessons

Although the linking order was part of the problem, it was not the whole problem.

Providing uninitialized global variables in a library doesn't always work, especially if there are no other definitions in the same source file.

As I noted in the chat, it is generally not a good idea to provide direct access to global variables. It would be better to provide functional interfaces to access (get and set) the variables.