provide replacement for memcpy@glibc_2.14 by LD_PRELOAD

Question

It's our old friend "version `GLIBC_2.14' not found". A customer actually needs to use a quite old Linux that only provides glibc version 2.11. And I'm stuck with precompiled libraries.

Linus LD_PRELOAD workaround doesn't work at all. I guess, it's because the library in my case explicitly requires memcpy@GLIBC_2.14.

So I tried another approach. First, find the function that needs the newer version.

$ export LD_LIBRARY_PATH=/home/kremers/experiment/lib
$ /home/kremers/experiment/runner
/home/kremers/experiment/runner: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.15' not found (required by /home/kremers/experiment/runner)
/home/kremers/experiment/runner: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/kremers/experiment/lib/libfoo.so)

Now, I tried to find the exact symbol, that is needed from GLIBC_2.14

$ readelf -s /home/kremers/experiment/lib/libfoo.so | grep GLIBC_2.14
       174: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND memcpy@GLIBC_2.14 (12)
      1242: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND memcpy@@GLIBC_2.14

I'm lucky. It's only memcpy! So I try to build my own memcpy@GLIBC_2.14, similar to [https://stackoverflow.com/a/33275588/1210825]

memcpy.c:

/* this was taken from https://bugzilla.redhat.com/show_bug.cgi?id=638477#c55
 * */

#include <sys/types.h>
void *memcpy(void *dst, const void *src, size_t size)
{
void *orig = dst;
asm volatile("rep ; movsq"
:"=D" (dst), "=S" (src)
:"0" (dst), "1" (src), "c" (size >> 3)
:"memory");
asm volatile("rep ; movsb"
:"=D" (dst), "=S" (src)
:"0" (dst), "1" (src), "c" (size & 7)
:"memory");
return orig;
}

memcpy.map:

GLIBC_2.14 {
   memcpy;
};

I build a shared library from that:

$ gcc -shared -fPIC -fno-builtin -c memcpy.c
$ gcc -shared -fPIC -Wl,--version-script memcpy.map -o libmemcpy-2.14.so memcpy.o -lc
$ cp libmemcpy-2.14.so ../lib/.

check, if symbols are present as needed:

$ readelf -sW /home/kremers/experiment/lib/libmemcpy-2.14.so | grep GLIBC
     4: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND __cxa_finalize@GLIBC_2.2.5 (3)
     5: 0000000000000000     0 OBJECT  GLOBAL DEFAULT  ABS GLIBC_2.14
    10: 000000000000061c   112 FUNC    GLOBAL DEFAULT   13 memcpy@@GLIBC_2.14
    53: 0000000000000000     0 OBJECT  GLOBAL DEFAULT  ABS GLIBC_2.14
    54: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND __cxa_finalize@@GLIBC_2.2.5

$ readelf -s /home/kremers/experiment/lib/libfoo.so | grep GLIBC_2.14
   174: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND memcpy@GLIBC_2.14 (12)
  1242: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND memcpy@@GLIBC_2.14

please note the difference libfoo requires memcpy@GLIBC_2.14 and memcpy@@GLIBC_2.14. libmemcpy only provides memcpy@GLIBC_2.14. And I don't know how to change that.

$ export LD_LIBRARY_PATH=/home/kremers/experiment/lib
$ export LD_PRELOAD=/home/kremers/experiment/lib/libmemcpy-2.14.so
$ /home/kremers/experiment/runner
/home/kremers/experiment/runner: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.15' not found (required by /home/kremers/experiment/runner)
/home/kremers/experiment/runner: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/kremers/experiment/lib/libfoo.so)

Nothing seems to have changed. I can only verify, that the libmemcpy is loaded:

$ ldd lib/libfoo.so
lib/libfoo.so: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by lib/libfoo.so)
lib/libfoo.so: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.15' not found (required by lib/libfoo.so)
        linux-vdso.so.1 =>  (0x00007fff46d99000)                 
        /home/kremers/experiment/lib/libmemcpy-2.14.so (0x00007fada31f7000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007fada2ee5000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fada2c8e000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fada2a78000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fada270a000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fada3669000)

Conclusion I get the impression, that this should work, but I'm missing a detail. My first guess is the memcpy@GLIBC_2.14 vs. memcpy@@GLIBC_2.14 mismatch.

PS I'm having a hard time searching for clues, because searching for "@" and "@@" don't seem well supported by bing or google. And while there is a lot of documentation for readelf, each document covers only a small portion. Usually leaving out what I was looking for. So please bear with me, if the solution is well documented somewhere.

Answer 1

The glibc dynamic linker currently does not support this. The original GNU symbol versioning specification requires several consistency checks, and these cannot be switched off and prevent adding completely new symbol versions to existing libraries via preloading.

First of all, you should simply interpose an unversioned symbol. All symbol versions will bind to it. Since you can arrange for no observable difference in behavior (just call memmove), you don't need fine-grained control here. A symbol memcpy@GLIBC_2.14 would not interpose anything, so the original specification requires that the dynamic linker checks that the soname of the symbol definition matches the soname contained in the symbol reference, and those won't match when preloading.

But this will not solve your problem because of another consistency check: The library, as identified by the soname in a symbol version reference, must use this symbol version to define any symbol. In short, binding for the symbol version itself is not lazy. It fails even if the symbol version is never used, either due to preloading or because the actual symbol reference is lazy and never bound.

We should probably remove the first check (we run into problems in glibc itself due to it), and add a knob to turn off the second check. But you would have to backport that change into your 2.11-derived glibc, and at that point, you can simply rebuild it and add an alias for memcpy with a versioned symbol memcpy@GLIBC_2.14. Sorry.