Is it possible to run unstripped binaries which loads unstripped libraries present in LD_LIBRARY_PATH

Question

In my target ARM embedded system, all applications and the dependent libraries are available only as stripped versions. Most of the binaries are present in /bin, /usr/bin paths and libraries present in /lib, /usr/lib path are all stripped.

But with these I was not able to run Valgrind and GDB. As both needs debug symbol information which is present in non-stripped version. So, I copied all the non-stripped (bins and libs) from my dev server to target system in a different path say(/tmp/test)

Bins -- /tmp/test/usr/bin, /tmp/test/bin
Libs -- /tmp/test/usr/lib, /tmp/test/lib

When I tried to run the unstripped binary, the process snmp_agent got crashed.

$LD_LIBRARY_PATH=/tmp/test/usr/lib/:/tmp/test/lib/ /tmp/test/usr/bin/snmp_agent -p 8000 &

Even I tried running dynamic linker as below but this time linker crashed.

$/tmp/test/lib/ld-2.24.so /tmp/test/usr/bin/snmp_agent -p 8000 &

Coredump when doing export and executing non-stripped bin:

root@myway:~# export LD_LIBRARY_PATH=/tmp/test/usr/lib/:/tmp/test/lib/; /tmp/test/usr/bin/snmp_agent -p 8000 &
[1] 21218
root@myway:~#
[1]+  Segmentation fault      (core dumped) /tmp/test/usr/bin/snmp_agent -p 8000
root@myway:~#

Getting coredump for default linux cmds after export:

root@myway:~# ls
Segmentation fault (core dumped)
root@myway:~# env
Segmentation fault (core dumped)
root@myway:~# pwd
/home/root
root@myway:~# date
Segmentation fault (core dumped)

Thanks @nidhoegger suggestion to use strace. Found that I missed symlink version of libs with strace output:

open("/tmp/test/usr/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/tmp/test/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\20=\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=427500, ...}) = 0
mmap2(NULL, 491640, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x76351000
mprotect(0x763b8000, 65536, PROT_NONE)  = 0
mmap2(0x763c8000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x67000) = 0x763c8000
close(3)                                = 0

Symlink lib in regular system lib path(stripped):

root@myway:~# ls -l /lib/libm.so.6
lrwxrwxrwx    1 root     root            12 Jun 22 11:30 /lib/libm.so.6 -> libm-2.24.so

I didnt copy the symlink version of unstripped libs to my path /tmp/test/usr/lib and /tmp/test/lib/

root@myway:~# ls -l /tmp/test/lib/libm.so.6
ls: /tmp/test/lib/libm.so.6: No such file or directory
root@myway:~# ls -l /tmp/test/lib/libm-2.24.so
-rwxrwx---    1 root     root       4885456 Jun 22 20:15 /tmp/test/lib/libm-2.24.so

Likewise my unstripped program is trying to load nearly 30 dependent libraries(verified this using ldd also) and all of which doesn't have symlink to original lib file. So the lack of symlink file is the reason behind the crash?

Usually I use `export $LD_LIBRARY_PATH...` I think exporting makes the updated value available to other processes. — kiner_shah, Jun 26 '21 at 06:33
@kiner_shah - Once I do `export`, execution of even default linux cmds like `ls, env,..` generates coredumps(**SEGFAULT**). — renga_in_stack, Jun 26 '21 at 07:09
The command should look like this: `export $LD_LIBRARY_PATH=/tmp/test/usr/lib/:/tmp/test/lib/; /tmp/test/usr/bin/snmp_agent -p 8000 &` Notice the `;` used to separate the two statements. Not sure if that will solve the issue, but that's how I run export command followed by an exe in a single command. — kiner_shah, Jun 26 '21 at 07:12
Run ldd on your executable to see what else it is picking up. If you are on hpux10, use chatr. — cup, Jun 26 '21 at 07:15
could you post the exact error message and the output of strace? — Nidhoegger, Jun 26 '21 at 07:24
Try using `export LD_LIBRARY_PATH=/tmp/test/usr/lib/:/tmp/test/lib/:$LD_LIBRARY_PATH`. This will ensure that the previous value is also used in case the required libraries aren't found in those two paths. — kiner_shah, Jun 26 '21 at 07:41

score 1 · Answer 1 · answered Jun 26 '21 at 08:54

You're effectively creating a partial chroot environment, and there are several tools which would probably help you in this situation, however I'll describe the algorithm that I use when doing this by hand.

Start with the binary, and the run-time loader that is used with the file. You already have a bin and lib subdirectory structure. Copy the ld.so into the lib directory, and the binary into the bin directory. Then start with this command:

env LD_LIBRARY_PATH=/tmp/test/lib:/tmp/test/usr/lib /tmp/test/lib/ld.so --list /tmp/test/bin/snmp_agent

It will give output like this (this is from a completely different system):

host$ /lib64/ld-linux-x86-64.so.2 --list /bin/ls
linux-vdso.so.1 (0x0000037433fc7000)
libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x0000037433958000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x0000037433567000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00000374332f5000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00000374330f1000)
/lib64/ld-linux-x86-64.so.2 (0x0000037433da2000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x0000037432ed2000)

Any libraries that are not found, or are referencing host libraries, and not libraries under /tmp/test/{lib,usr/lib} are ones that should be copied into /tmp/test/lib.

Repeat until there are no unresolved libraries.

For execution, you do:

env LD_LIBRARY_PATH=/tmp/test/lib:/tmp/test/usr/lib /tmp/test/lib/ld.so /tmp/test/bin/snmp_agent -p 8000

Next comes the 'runner' - I generally rename the binaries to binary.real, and keep them all in a single bin directory (this makes the script a lot easier) - you would need to adapt this runner if you choose to keep a bin and usr/bin, an use the following:

#!/bin/bash -p

dir=$(dirname "${BASH_SOURCE[0]}")
env LD_LIBRARY_PATH="$dir/../lib:$dir/../usr/lib" "$dir/../lib/ld.so" "${0}.real" "$@"

Now you put copies/links of this file for all the programs you want to run.

If you're ld.so is newer than v2.33, then you can use the -argv0 option when invoking the ld.so directly to give it the proper argv0 name:

env LD_LIBRARY_PATH="$dir/../lib:$dir/../usr/lib" "$dir/../lib/ld.so" -argv0 "$0" "$0.real" "$@"

Employed Russian · Accepted Answer · 2021-07-01T07:13:40.157

So the lack of symlink file is the reason behind the crash?

No, the reason behind the crash is that your unstripped binaries are coming from a different build.

If the difference was only due to stripping, you would be able to mix and match stripped and unstripped binaries freely, but you clearly can't.

The reason why all parts of GLIBC (the loader ld-linux, libc.so.6, libpthread.so.0, libdl.so.2, etc) must match exactly (must come from the same build) is explained here.

Is it possible to run unstripped binaries which loads unstripped libraries present in LD_LIBRARY_PATH

2 Answers2