How to link libraries when cross compiling

Question

I'm cross compiling open VMWare tools. I pre-compiled glib and i'm setting the PKG_CONFIG_PATH variable to link them. I'm getting the following errors in the link stage.

libtool: link: warning: library `/u/git/extlib_vmtools/usr/local/lib/libgthread-2.0.la' was moved.
/bin/grep: /usr/local/lib/libglib-2.0.la: No such file or directory
/bin/sed: can't read /usr/local/lib/libglib-2.0.la: No such file or directory
libtool: link: `/usr/local/lib/libglib-2.0.la' is not a valid libtool archive
make[2]: *** [libhgfs.la] Error 1
make[2]: Leaving directory `/u/git/extlib_vmtools/src/open-vm-tools-11.0.0-14549434/libhgfs'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/u/git/extlib_vmtools/src/open-vm-tools-11.0.0-14549434'

I followed this helpful post (DESTDIR and PREFIX of make) but I think i'm missing something. I didn't set the --prefix for any module, since I want to use the default direcotry structure when deploying.

glib compilation (lower module):

./configure --host=${CROSS_COMPILE_HOST}
make
make install DESTDIR=/home/xport/

open-vmware-tools compilation (upper module):

export PKG_CONFIG_SYSROOT_DIR="/home/xport/"
export PKG_CONFIG_PATH="/home/xport/usr/local/lib/pkgconfig"
autoreconf -i
./configure --host=${CROSS_COMPILE_HOST} --enable-lib64=no --without-kernel-modules --without-pam --disable-vgauth --without-x --without-gtk3 --without-gtk2 --without-gtkmm3 --without-gtkmm --enable-resolutionkms=no --enable-deploypkg=no
make
make install DESTDIR=/home/xport # <-- I don't even get here

Before I set the PKG_CONFIG_PATH variable, the second make failed due to missing .h files... So I know that .pc linkage works. What am I missing ? Thanks !

When you run `configure` you need to specify the *target* to build for, the host will by default to the local host system. — Some programmer dude, Dec 17 '19 at 16:02
@SomeProgrammerDude - close. Use `--build`, not `--target`. `--target` is used for building the compiler for an arch. Very confusing... — jww, Dec 17 '19 at 16:38

jww · Accepted Answer · 2019-12-17T17:45:14.503

./configure --host=${CROSS_COMPILE_HOST} ...

You need to set both --build and --host dues to an Autoconf bug. Assuming you are building for ARMv7l, something like:

./configure --host=$(config.guess) --build=armv7l-unknown-linux-gnueabihf

The following looks OK to me, assuming /home/xport/usr/local/lib/pkgconfig is valid and /home/xport/usr/local is the location of the include/ and lib/ files for the arch.

export PKG_CONFIG_PATH="/home/xport/usr/local/lib/pkgconfig"

I'm not sure about what follows. It is also missing a --build. And I am used to seeing a --sysroot when cross-compiling:

./configure --host=${CROSS_COMPILE_HOST} \
    --enable-lib64=no --without-kernel-modules --without-pam --disable-vgauth \
    --without-x --without-gtk3 --without-gtk2 --without-gtkmm3 --without-gtkmm \
    --enable-resolutionkms=no --enable-deploypkg=no

CFLAGS and CXXFLAGS should probably include --sysroot. Maybe something like:

./configure --host=$(config.guess) --build=armv7l-unknown-linux-gnueabihf \
    --sysroot="/home/xport/usr/local" --enable-lib64=no --without-kernel-modules ...

Or:

    CFLAGS="--sysroot=/home/xport/usr/local" \
    CXXFLAGS="--sysroot=/home/xport/usr/local" \
./configure --host=$(config.guess) --build=armv7l-unknown-linux-gnueabihf \
    --enable-lib64=no --without-kernel-modules ...

DESTDIR is intended for staging. This looks OK:

make install DESTDIR=/home/xport/

If you intend to run from the /home/xport/ directory, then you should consider adding the following to LDFLAGS:

# 64-bit Fedora
-Wl,--enable-new-dtags -Wl,-R,'$$ORIGIN/../lib64'
# Most others
-Wl,--enable-new-dtags -Wl,-R,'$$ORIGIN/../lib'

So maybe something like:

    CFLAGS="--sysroot=/home/xport/usr/local" \
    CXXFLAGS="--sysroot=/home/xport/usr/local" \
    LDFLAGS="-Wl,--enable-new-dtags -Wl,-R,'$$ORIGIN/../lib'" \
./configure --host=$(config.guess) --build=armv7l-unknown-linux-gnueabihf \
    --enable-lib64=no --without-kernel-modules ...

$ORIGIN-based runpaths is what makes staged installs with DESTDIR work.

The double dollar sign in $$ORIGIN is due to Makefiles. The double dollar sign is the way to escape the dollar sign so it passes through the makefile properly.

Also see How to include correctly -Wl,-rpath,$ORIGIN linker argument in a Makefile?

config.guess provides you with your Autoconf triplet:

$ /usr/share/libtool/build-aux/config.guess
x86_64-pc-linux-gnu

If you don't have a config.guess on-path, then check for it in /usr/share:

$ find /usr/share -name config.guess
/usr/share/misc/config.guess
/usr/share/libtool/config/config.guess
/usr/share/automake-1.14/config.guess

Also see 2.2.8 Cross-Compilation in the Autoconf manual:

To cross-compile is to build on one platform a binary that will run on another platform. When speaking of cross-compilation, it is important to distinguish between the build platform on which the compilation is performed, and the host platform on which the resulting executable is expected to run. The following configure options are used to specify each of them:

--build=build

The system on which the package is built.

--host=host

The system where built programs and libraries will run.

And a little further down:

The --host and --build options are usually all we need for cross-compiling. The only exception is if the package being built is itself a cross-compiler: we need a third option to specify its target architecture.

--target=target

When building compiler tools: the system for which the tools will create output.

Regarding the comment "I don't even get here":

make
make install DESTDIR=/home/xport # <-- I don't even get here

You need to show the compile error you are encountering when you run make.

OK Testing... config.guess=x86_64-unknown-linux-gnu and CROSS_COMPILE_HOST=i686-linux. Is that OK and your comment still applies ? Because glib (one of the dependencies) fails the configure pre-check. Thanks ! — user2491552, Dec 18 '19 at 17:48
Also, /home/xport is the stating directory in which I would like to exported the compilation output. I will tgz this directory and extract it in / on my target machine. So sysroot is not needed I think. — user2491552, Dec 18 '19 at 17:53
@user2491552 - Based on the `CROSS_COMPILE_HOST` comment, you should probably use `CROSS_COMPILE_HOST=i686-unknown-linux-gnu` and `--host=i686-unknown-linux-gnu` if you are building to run on 32-bit i686. — jww, Dec 18 '19 at 20:12
@user2491552 - Based on my experience with cross-compiling, you want to set a `--sysroot`. That is a build time option, and tells the toolchain where to find tools, headers and libraries. `--sysroot` does not affect runtime. For runtime, you want to compile and link with `-Wl,--enable-new-dtags -Wl,-R,'$$ORIGIN/../lib'`. That allows you to move the program around on filesystems. — jww, Dec 18 '19 at 20:16
@user2491552 - At this point, I think you should reboot. Use the proper triplets for `--build`, `--host` and `CROSS_COMPILE_HOST`. Set `CFLAGS`, `CXXFLAGS` and `LDFLAGS`. Build everything again. When you have a new problem, ask a new question. — jww, Dec 18 '19 at 20:18

How to link libraries when cross compiling

1 Answers1