Trying to run a cross-compiled executable on target device fails with: No such file or directory

Question

I've got caught in the not-so-sunny world of cross-compilation.

I'm trying to compile a simple hello world application for my BeagleBone Black (which runs a TI Cortex-A8 processor).

First of all, I compiled and ran successfully the hello world application on x86 with gcc

Then I changed my compilation settings to the following:

arm-linux-gnueabi-gcc -c -O0 -g3 -Wall main.c -o bin/obj/main.o
arm-linux-gnueabi-gcc bin/obj/main.o -o bin/hello_world

I transferred the file via SCP to the BeagleBone, and set executable permissions with chmod +x hello_world

Upon running it (./hello_world), my only response is:

-bash: ./hello_world: No such file or directory

The output of file matches that of /sbin/init as I would expect:

$ file hello_world
hello_world: ELF 32-bit LSB executable, ARM, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=0x24b659b7a41fe043a6f4649d4ebfb5e692ebf0c7, not stripped
$ file /sbin/init
/sbin/init: ELF 32-bit LSB executable, ARM, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.26, BuildID[sha1]=0xd21f6957ec031a27d567b3d5e6aa14b9e0c30c37, stripped

The result of ldd is:

$ ldd hello_world
    not a dynamic executable

I tried adding a suitable platform and CPU type, changing my compilation to:

arm-linux-gnueabi-gcc -c -O0 -g3 -Wall -march=armv7-a -mtune=cortex-a8  main.c -o bin/obj/main.o
arm-linux-gnueabi-gcc bin/obj/main.o -o bin/hello_world

This initially started giving me a new error: Text file busy, but I have since been unable to get that error back again as it now returns No such file or directory. I'm guessing that particular attempt was just a bad transfer or something.

Answer 1

Since nobody from the comments posted the answer, I guess I get the pleasure ;)

No such file or directory comes from when the kernel tries to invoke the dynamic linker specified by the ELF executable's .interp field, but no such file exists.

The .interp field can be found with the following command:

objdump -j .interp -s ./hello_world

In the case of this example, the executable's .interp field was /lib/ld-linux.so.3, but the name of the dynamic linker on the BeagleBone Black is /lib/ld-linux-armhf.so.3.

This happened because the program was compiled with a slightly different toolchain to the one required for the platform. It should be arm-linux-gnueabihf-* rather than arm-linux-gnueabi-*.

The difference between the two is that the Cortex-A8 uses specific floating point registers with the hard-float version (armhf) of the EABI, but the original EABI (armel) uses integer registers for passing around floating point numbers. As a result, armel programs will run on armhf (provided the dynamic linker is set to the correct path!), but not vice versa.

Simply adding a symbolic link ln -s /lib/ld-linux-armhf.so.3 /lib/ld-linux.so.3 is enough to resolve this issue, but the correct fix is to use the right toolchain when compiling the program in the first place.

Answer 2

I had the same problem. I downloaded and installed gcc-arm-linuc-gnueabihf package to my Ubuntu PC from the Ubuntu repository using apt-get. Then I compiled a helloworld test program and downloaded it using sftp to my BeagleBone.

Trying to run the program on BBB gave the error: "No such file or directory"

Using objdump I found that the .interp field in the ELF executable was /lib/ld_linux_armhf.so.3. My BBB had dynamic linker /lib/ld-linux.so.3.

I created a symbolic link on the BBB:

ln -s /lib/ld-linux.so.3 /lib-linux-armhf.so.3

Now the cross-compiled application works on BBB. The BBB is running the original Angstrom distribution.

This is not the ideal fix. Now I need to either confgure the toolchain in Ubuntu to add the correct dynamic linker name to the app, or update the BBB to have a dynamic linker specified in the toolchain.

I assume the error message is due to the dynamic linker file not found, not that the application doesn't exist.

Answer 3

How to identify the problem?

file cross_compiled_executable

Contains something like:

interpreter /lib/ld-uClibc.so.0

and the problem is that that file does not exist on the target.

How to solve the problem?

Use a proper compiler, either:

• the person who created the disk image must provide you the cross compiler or tell you exactly how to build it, e.g. with crosstool-ng. For BeagleBone asked at: Toolchain to crosscompile Applications for BBB
• compile your own image and cross compiler, e.g. with Buildroot. Here is a generic QEMU example. Buildroot has BeagleBone support.

• use a native compiler on the target. But generally targets are much slower than your host, and space constrained, so you likely don't want to do this.

  You might also be able to use a functional emulator such as QEMU to build, and then only run the programs on a slower platform, e.g. gem5 or a slow board.

Just hacking up the interpreter is potentially not enough, notably you have to ensure binary compatibility between the program and the target libc, or program and kernel interfaces (syscalls, /proc, etc.) if you try to use -static (the target kernel might be too old and not contain the required interfaces). The only robust solution is to use the correct toolchain.