83

Linux binaries are usually dynamically linked to the core system library (libc). This keeps the memory footprint of the binary quite small but binaries which are dependent on the latest libraries will not run on older systems. Conversely, binaries linked to older libraries will run happily on the latest systems.

Therefore, in order to ensure our application has good coverage during distribution we need to figure out the oldest libc we can support and link our binary against that.

How should we determine the oldest version of libc we can link to?

Jeffrey Bosboom
  • 13,313
  • 16
  • 79
  • 92
Gearoid Murphy
  • 11,834
  • 17
  • 68
  • 86

4 Answers4

103

Work out which symbols in your executable are creating the dependency on the undesired version of glibc.

$ objdump -p myprog
...
Version References:
  required from libc.so.6:
    0x09691972 0x00 05 GLIBC_2.3
    0x09691a75 0x00 03 GLIBC_2.2.5

$ objdump -T myprog | fgrep GLIBC_2.3
0000000000000000      DF *UND*  0000000000000000  GLIBC_2.3   realpath

Look within the depended-upon library to see if there are any symbols in older versions that you can link against:

$ objdump -T /lib/libc.so.6 | grep -w realpath
0000000000105d90 g    DF .text  0000000000000021 (GLIBC_2.2.5) realpath
000000000003e7b0 g    DF .text  00000000000004bf  GLIBC_2.3   realpath

We're in luck!

Request the version from GLIBC_2.2.5 in your code:

#include <limits.h>
#include <stdlib.h>

__asm__(".symver realpath,realpath@GLIBC_2.2.5");

int main () {
    realpath ("foo", "bar");
}

Observe that GLIBC_2.3 is no longer needed:

$ objdump -p myprog
...
Version References:
  required from libc.so.6:
    0x09691a75 0x00 02 GLIBC_2.2.5

$ objdump -T myprog | grep realpath
0000000000000000      DF *UND*  0000000000000000  GLIBC_2.2.5 realpath

For further information, see http://web.archive.org/web/20160107032111/http://www.trevorpounds.com/blog/?p=103.

Community
  • 1
  • 1
Sam Morris
  • 1,858
  • 1
  • 17
  • 18
  • 9
    I would also add that often there's only one or two symbols causing a dependency on a new glibc version, so if like me you are worried you're going to have to list hundreds of symbols to remove a dependency, you won't. – Malvineous Feb 18 '12 at 07:51
  • 2
    dietlibc is also worth looking at. – Gearoid Murphy May 31 '12 at 10:37
  • 1
    Or if you can link statically (i.e. your code is not a plugin and won't use plugins), you can also look at musl-libc. I have found oftentimes that the statically linked programs with musl-libc are smaller than their dynamically linked glibc counterparts. – 0xC0000022L Dec 15 '15 at 20:08
  • @GearoidMurphy about five years later, this comment came around to help me out in a major way. :) – Tango Bravo Feb 19 '17 at 16:29
  • 9
    This just forces linking to an older symbol version, without any regard for the reasons that the newer version was created. Doing this in general could have all kinds of problems. If using old symbol versions is needed, compilation against headers of that old version is needed as well. This approach might work with some really simple calls that don't have pointers to system defined structs, but still is not advisable. – textshell Sep 12 '19 at 20:54
15

Unfortunately, @Sam's solution doesn't work well in my situation. But according to his way, I found my own way to solve that.

This is my situation:

I'm writing a C++ program using the Thrift framework(it's an RPC middleware). I prefer static link to dynamic link, so my program is linked to libthrift.a statically instead of libthrift.so. However, libthrift.a is dynamically linked to glibc, and since my libthrift.a is build on my system with glibc 2.15, my libthrift.a uses memcpy of version 2.14(memcpy@GLIBC_2.14) provided by glibc 2.15.

But the problem is that our server machines have only the glibc version 2.5 which has only memcpy@GLIBC_2.2.5. It is much lower than memcpy@GLIBC_2.14. So, of course, my server program can't run on those machines.

And I found this solusion:

  1. Using .symver to obtain the ref to memcpy@GLIBC_2.2.5.

  2. Write my own __wrap_memcpy function which just calls memcpy@GLIBC_2.2.5 directly.

  3. When linking my program, add -Wl,--wrap=memcpy option to gcc/g++.

The code involved in steps 1 and 2 is here: https://gist.github.com/nicky-zs/7541169

nicky_zs
  • 3,633
  • 1
  • 18
  • 26
  • 2
    The problem appears to be with `_FORTIFY_SOURCE`, which defaults to 1 on newer GCC versions at certain optimization levels. With that enabled, some of the functions are masked. For me undefining and subsequently redefining it to 0 makes the above solution work (GCC 4.8.4). – 0xC0000022L Dec 15 '15 at 20:05
  • 9 years later, it saved me a lot of time. Many thanks! – Ayush Rawal Sep 05 '22 at 10:22
12

To do this in a more automated fashion, you can use the following script to create a list of all the symbols that are newer in your GLIBC than in a given version (set on line 2). It creates a glibc.h file (filename set by the script argument) which contains all the necessary .symver declarations. You can then add -include glibc.h to your CFLAGS to make sure it gets picked up everywhere in your compilation.

This is sufficient if you don't use any static libraries that were compiled without the above include. If you do, and you don't want to recompile, you can use objcopy to create a copy of the library with the symbols renamed to the old versions. The second to bottom line of the script creates a version of your system libstdc++.a that will link against the old glibc symbols. Adding -L. (or -Lpath/to/libstdc++.a/) will make your program statically link libstdc++ without linking in a bunch of new symbols. If you don't need this, delete the last two lines and the printf ... redeff line.

#!/bin/bash
maxver=2.9
headerf=${1:-glibc.h}
set -e
for lib in libc.so.6 libm.so.6 libpthread.so.0 libdl.so.2 libresolv.so.2 librt.so.1; do
objdump -T /usr/lib/$lib
done | awk -v maxver=${maxver} -vheaderf=${headerf} -vredeff=${headerf}.redef -f <(cat <<'EOF'
BEGIN {
split(maxver, ver, /\./)
limit_ver = ver[1] * 10000 + ver[2]*100 + ver[3]
}
/GLIBC_/ {
gsub(/\(|\)/, "",$(NF-1))
split($(NF-1), ver, /GLIBC_|\./)
vers = ver[2] * 10000 + ver[3]*100 + ver[4]
if (vers > 0) {
    if (symvertext[$(NF)] != $(NF-1))
        count[$(NF)]++
    if (vers <= limit_ver && vers > symvers[$(NF)]) {
        symvers[$(NF)] = vers
        symvertext[$(NF)] = $(NF-1)
    }
}
}
END {
for (s in symvers) {
    if (count[s] > 1) {
        printf("__asm__(\".symver %s,%s@%s\");\n", s, s, symvertext[s]) > headerf
        printf("%s %s@%s\n", s, s, symvertext[s]) > redeff
    }
}
}
EOF
)
sort ${headerf} -o ${headerf}
objcopy --redefine-syms=${headerf}.redef /usr/lib/libstdc++.a libstdc++.a
rm ${headerf}.redef
patstew
  • 1,806
  • 17
  • 21
  • Why the `sort`? – Otheus Oct 13 '17 at 15:32
  • It's not really necessary, it just makes the file more reproducible if you want to check it in to source control or prevent your build system rebuilding everything – patstew Oct 14 '17 at 16:00
  • This `objcopy --redefine-syms` solution doesn't actually work, and it took me hours to find why. objcopy tries to do a string match-replace for the given symbols, but the versions of the symbols are _not_ expressed as strings . objcopy simply doesn't work on them -- the versions are expressed in a lookup table, which objcopy 2.20.50 does not attempt to modify. Ever. There _is_ a "default" symbol which can be changed this way -- `symbol@@1.2.3` can be changed. – Otheus Oct 19 '17 at 17:50
  • I thought I had that working, but I mustn't have tested it properly if that's the case, sorry. I didn't end up using that part, I settled on recompiling my deps since it seems less likely to introduce runtime problems. – patstew Oct 19 '17 at 19:35
  • 1
    Great answer, thank you so much! Helps when your build machine is newer than target. In my case, I'm building on newest Ubuntu focal (glibc 2.31-0ubuntu9.1) for Debian buster (2.28-10). If maxver is too low, it will produce a relocation error like "symbol clock_gettime version GLIBC_2.2.5 not defined in file libc.so.6 with link time reference" but with right maxver (2.20) it produces a working binary. This should really be part of glibc manual! – ArticIceJuice Oct 26 '20 at 19:12
  • It's also worth checking out crosstool-ng to build a toolchain with old glibc and new GCC/libstdc++ – patstew Oct 26 '20 at 19:21
  • @patstew this was a case of "build on your laptop - run on an embedded device". I didn't want to set up a separate build VM, chroot or anything like it. Software is a fork of ffmpeg to stream directly from H264-enabled camera https://github.com/articice/FFmpeg/tree/uvcvideo-patch-rebase – ArticIceJuice Oct 26 '20 at 22:28
  • @ArticIceJuice that is exactly the error I'm getting, do you know why this happens? If I use a higher maxver value that works in a current version of glibc, could it return this error in future versions? Thanks! – ciclopez Mar 03 '21 at 13:40
  • I think they moved clock_gettime from librt to libc around 2.17 or something, make sure you have -lrt in your linker args. If that's not it maybe that move is causing other problems due to the header files or something? – patstew Mar 03 '21 at 13:59
6

glibc 2.2 is a pretty common minimum version. However finding a build platform for that version may be non-trivial.

Probably a better direction is to think about the oldest OS you want to support and build on that.

Douglas Leeder
  • 52,368
  • 9
  • 94
  • 137