Question
Description
Im trying to compile something and when that required libblkid, libudev, and pthreads. When I compile and link against liblkid and libudev I predictably get some errors about how it cant find pthreads functions. However, when I compile and link against liblkid, libudev, and pthreads, the linker complains that it can't find anything from liblkid and libudev.
Errors
When compiling and linking without pthread
/usr/bin/ld: ../zfs/lib/libzfs_core.a(thread_pool.o): undefined reference to symbol 'pthread_attr_setaffinity_np@@GLIBC_2.3.4'
/usr/bin/ld: /lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line
When compiling and linking with pthread
/usr/bin/ld: ../zfs/lib/libzfs.a(libzfs_util.o): in function `zfs_nicestrtonum':
/tmp/zfs/lib/libzfs/libzfs_util.c:1559: undefined reference to `pow'
/usr/bin/ld: ../zfs/lib/libzfs.a(libzfs_changelist.o): in function `changelist_prefix':
/tmp/zfs/lib/libzfs/libzfs_changelist.c:107: undefined reference to `uu_avl_walk_start'
/usr/bin/ld: /tmp/zfs/lib/libzfs/libzfs_changelist.c:110: undefined reference to `uu_avl_walk_next'
/usr/bin/ld: /tmp/zfs/lib/libzfs/libzfs_changelist.c:152: undefined reference to `uu_avl_walk_end'
/usr/bin/ld: ../zfs/lib/libzfs.a(libzfs_changelist.o): in function `changelist_postfix':
/tmp/zfs/lib/libzfs/libzfs_changelist.c:186: undefined reference to `uu_avl_last'
...
Replicating my work
I did this while compiling zfs from source so making a minimal example is beyond me. However, I bundled everything into a docker container.
docker run -ti --rm failuregod/example to run the container
I used cmake for this example and the docker container will drop you in the build folder.
make nothread VERBOSE=1 builds without pthread and will print the gcc command that was run.
make thread VERBOSE=1 builds with pthread and will print the gcc command that was run.
This is not a duplicate
This is not a duplicate question because it is not asking "what does undefined reference mean". This question is asking "how do I link to this version of libzfs that I built from source." So answers like "well just include the right libraries in the right order" are unhelpful. A good answer would tell me how to make a CMakeLists.txt and Findzfs.cmake that works.
Answer
TLDR
I figured out-ish how to get it to link and stuff and I used pkg_check_modules in cmake.
The saga
I'm gonna go ahead and assume the next punk that finds this post knows only as much as I did when I started this journey.
Pkg-Config
As it so happens theres a type of file called a package config that takes the name libzfs.pc (or in general libname.pc). This package config file describes what dependencies libzfs has, what flags it needs to compile, and what flags it needs to link. Here's the libzfs.pc file for you to get a sense of it:
prefix=/root/zfs
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include
Name: libzfs
Description: LibZFS library
Version: 2.1.0
URL: https://github.com/openzfs/zfs
Requires: libzfs_core
Requires.private: libcrypto zlib
Cflags: -I${includedir}/libzfs -I${includedir}/libspl
Libs: -L${libdir} -lzfs -lnvpair
Libs.private: -luutil -lm -pthread
This file tells me, for example, that I need to be using -I /root/zfs/include/libzfs and -I /root/zfs/include/libspl when I'm compiling against libzfs.
There's a tool (that's called pkg-config and that's not installed by default) that reads these configs and lets you ask things like "what are the include flags I need for this library" (pkg-config --cflags-only-I <lib>), or "what are the link flags I need for libzfs if I'm linking it statically" (pkg-config --static --libs libzfs), etc. This tool works by reading the .pc package config files. It looks in a couple standard locations and, of course, the zfs I built in my home directory is not a standard location.
In zfs the package configs are found in prefix/lib/pkgconfig (at least in 2.1.0. In 0.8.3 it's in a prefix/share/pkgconfig) where prefix in this case is /root/zfs. So to allow pkg-config to find libzfs and everything else, I had to set PKG_CONFIG_PATH=/root/zfs/lib/pkgconfig.
Cmake
This tool is only half of what's needed. If you run the docker container - that I so graciously provided - you'll see that I'm using cmake. If you read the CMakeLists.txt you'll see that I call find_package and provide my own Findblkid.cmake and Findudev.cmake (or well they aren't mine, they're from ceph). As for the actual zfs portion I included everything.
There is another way.
Cmake is wise and provides a function called pkg_check_modules that uses pkg-config behind the scenes to figure out everything that's needed to link a package.
So I made a new CMakeLists.txt file:
cmake_minimum_required(VERSION 3.1)
project(PackageTest C)
add_executable(prog example.c)
find_package(PkgConfig REQUIRED)
pkg_check_modules(ZFS REQUIRED IMPORTED_TARGET libzfs)
target_link_libraries(prog PRIVATE PkgConfig::ZFS)
target_compile_definitions(prog PRIVATE _GNU_SOURCE)
As far as I can tell _GNU_SOURCE should be in libzfs.pc because the program wont compile without it. So, as far as I can tell, the maintainers made a mistake and not including it in libzfs.pc is a bug.
