Linker error: "relocation R_X86_64_PC32 against symbol `xmlFree' ... recompile with -fPIC" but offending library was already compiled with -fPIC

Question

I am trying to build a shared library. My aim is to pull all dependencies as static libraries into a single shared library. My understanding is that this can be done with the -Wl,--whole-archiveflag. Here is a snippet of my CMake script responsible for configuring the shared library.

    # shared library
    add_library(semsim SHARED "${SEMSIM_HEADERS}" "${SEMSIM_SOURCES}") # created the shared library
    #fPIC for linux shared library strs
    set_property(TARGET semsim PROPERTY POSITION_INDEPENDENT_CODE ON) #turn fPIC on 
    target_compile_options(semsim PRIVATE -Wl,--whole-archive) # enable pulling static libraries into shared library

The error I'm getting whilst trying to compile (verbose mode is on) is:

/usr/bin/c++ -fPIC -std=c++14 -g  -shared -Wl,-soname,libsemsim.so -o libsemsim.so CMakeFiles/semsim.dir/CurlGet.cpp.o CMakeFiles/semsim.dir/RDFNode.cpp.o CMakeFiles/semsim.dir/Subject.cpp.o CMakeFiles/semsim.dir/Predicate.cpp.o CMakeFiles/semsim.dir/Resource.cpp.o CMakeFiles/semsim.dir/Triple.cpp.o CMakeFiles/semsim.dir/SemsimUtils.cpp.o CMakeFiles/semsim.dir/MetaID.cpp.o CMakeFiles/semsim.dir/XmlAssistant.cpp.o CMakeFiles/semsim.dir/Reader.cpp.o CMakeFiles/semsim.dir/Editor.cpp.o CMakeFiles/semsim.dir/Writer.cpp.o CMakeFiles/semsim.dir/RDF.cpp.o CMakeFiles/semsim.dir/Participant.cpp.o CMakeFiles/semsim.dir/PhysicalEntity.cpp.o CMakeFiles/semsim.dir/PhysicalPhenomenon.cpp.o CMakeFiles/semsim.dir/PhysicalProcess.cpp.o CMakeFiles/semsim.dir/PhysicalPropertyResource.cpp.o CMakeFiles/semsim.dir/PhysicalForce.cpp.o CMakeFiles/semsim.dir/Query.cpp.o CMakeFiles/semsim.dir/SemsimCombineArchive.cpp.o CMakeFiles/semsim.dir/Triples.cpp.o ../../third_party/libCombine-0.2.3/INSTALL/lib/libcombine-static.a ../../third_party/zipper/INSTALL/lib/libZipper-static.a ../../third_party/zlib-1.2.11/INSTALL/lib/libz.a /usr/local/lib/libbz2.a /usr/local/lib/libxml2.a -ldl -lbz2 -lz -lcurl -lxslt 
/usr/bin/ld: /usr/local/lib/libxml2.a(uri.o): relocation R_X86_64_PC32 against symbol `xmlFree' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status

However, according to this question, /usr/local/lib/lxml2.a was already compiled fPIC:

(base) ciaran@DESKTOP:/usr/local/lib$ ls
cmake            libcharset.so.1.0.0  libraptor2.a         librasqal.la        librdf.so        libxml2.so.2
libbz2.a         libcurlcpp.a         libraptor2.la        librasqal.so        librdf.so.0      libxml2.so.2.9.10
libcharset.a     libiconv.la          libraptor2.so        librasqal.so.3      librdf.so.0.0.0  pkgconfig
libcharset.la    libiconv.so          libraptor2.so.0      librasqal.so.3.0.0  libxml2.a        python2.7
libcharset.so    libiconv.so.2        libraptor2.so.0.0.0  librdf.a            libxml2.la       python3.6
libcharset.so.1  libiconv.so.2.6.1    librasqal.a          librdf.la           libxml2.so       xml2Conf.sh
(base) ciaran@DESKTOP:/usr/local/lib$ readelf -d libxml2.a | grep TEXT
(base) ciaran@DESKTOP:/usr/local/lib$

This leads me to believe that perhaps I've misunderstood the error message - could somebody shed some light on what's going on here?

Answer 1

You haven't misunderstood the error message, or at least not significantly.

However, according to this question, /usr/local/lib/lxml2.a was already compiled fPIC

You conclude this because the command:

/usr/local/lib$ readelf -d libxml2.a | grep TEXT

outputs nothing, which shows that you chose the second most popular answer to that question, although at this writing it is not nearly as up-voted as the most popular answer

I guess you did that because the most popular answer tells you how to test an object file for PIC-ness, and you want to a test a library, as the second most popular answer does. Or perhaps because you tried the most popular answer first and it indicated that your libxml2.a was PIC.

But the second most popular answer tests a shared library, and you use it to test the object files in a static library. A shared library is very different from an object file, or a static library of such, and the question to which this answer was given asks how to test the PIC-ness of an object file. So this answer does not answer the question: really it suggests how to test whether a file that has a name like a shared library actually is a shared library.

The most popular answer does answer the question, and if was a correct answer then it would be the right one for you too, because what the linker has diagnosed:

/usr/local/lib/libxml2.a(uri.o): relocation R_X86_64_PC32 against symbol `xmlFree' \
can not be used when making a shared object; recompile with -fPIC

is that the object file uri.o archived in libxml2.a is not PIC.

But the most popular answer, notwithstanding its popularity, is invalid. The PIC-ness test that it proposes - with no supporting argument - can misidentify non-PIC object files as PIC (as perhaps you observed).

None of the answers to that question is a correct answer for you, and it doesn't really matter why. The system linker itself is the ultimate authority on whether an object file is or is not position-independent. Any one-liner test other than attempting to link the object file into a DSO is just an attempt to second-guess the linker: if its verdict differs from the linker's it implies that the test is broken, not the linker.

And you already know the linker's verdict. It has tried to link libxml2.a(uri.o) into a DSO and found that it can't, because the it contains a non-position-independent relocation record.

The /usr/local/lib/libxml2.a you've got was built on the traditional default assumption that the object files archived therein need not be compiled to position-independent code (with -fPIC) because the static library would only be input to the linkage of non-position-independent executables. If you want to link some PI binary, then you'll link it with the shared libary libxml2.so, which is PI by definition. Your libxml2.a was furthermore built with a compiler that does not emit PI object code by default. It is possible that your current compiler still has that increasingly antiquated trait, but you don't need to find out.

You need to replace your local libxml2 install with one in which the object files in libxml2 have been compiled with fPIC. If you already know how to do that you can skip the rest of this and get on with it.

If you have somewhere got the source package of libxml2 that was built and installed and you want to stick with that revision, then cd into its root directory and run:

make uninstall

Otherwise, remove the installation by deleting, as root, all files and symlinks matching /usr/local/lib/libxml2 and the directory /usr/local/include/libxml2

If you want to stick with a source package you've already got (where you ran make uninstall), then, in its root directory, run:

make distclean

to restore to its pristine state.

If you don't have a source package you want to stick with, then clone or download-and-extract the latest from https://gitlab.gnome.org/GNOME/libxml2/, then cd into the root directory and run:

./autogen.sh

to generate the build-system.

Whatever you've done thus far, in the package root directory then run:

./configure CFLAGS=-fPIC [any other non-default configuration options]

If that completes successfully, then run:

make

If that completes successfully, then as root run:

make install

If that completes successfully, then /usr/local/lib/libxml2.a will be recreated containing PI object files and you will be able to link your shared library against it.

If you are unsure about [any other non-default configuration options] for the ./configure command, then run:

./configure -h

beforehand for help, and/or seek advice.