Why does changing the linking order fix some linking errors on one system?

Question

So I had this strange behavior with GitLab CI. I got it working but now I am wondering why it works.

First of all I was starting with GitLab CI. I got a local runner with docker on my machine (Arch Linux), so that I can test without pushing and waiting. I wrote a test with the googletest framework (Just an assert true). I triggered the script locally and everything worked. All tests passed in the local docker image.

So now, when everything was working, I pushed to the repository and a runner took the job. This ran on Ubuntu 16.04. Now it compiled and after the execution it was throwing a segmentation fault.

I got to debugging at the Ubuntu system and after a while I switched the linking order of two libraries:

From:

target_link_libraries(${PROJECT_NAME}_test PRIVATE Threads::Threads -lglog -lpthread
    ${QT_LIBRARIES}
    ${PROJECT_BINARY_DIR}/googletest-build/googlemock/libgmock.a
    ${PROJECT_BINARY_DIR}/googletest-build/googlemock/gtest/libgtest.a
    ${OpenCV_LIBRARIES}
)

To:

target_link_libraries(${PROJECT_NAME}_test PRIVATE Threads::Threads -lglog -lpthread
    ${QT_LIBRARIES}
    ${OpenCV_LIBRARIES}
    ${PROJECT_BINARY_DIR}/googletest-build/googlemock/libgmock.a
    ${PROJECT_BINARY_DIR}/googletest-build/googlemock/gtest/libgtest.a
)

I am using CMake for the build.

Both PCs are running the same version of docker (17.12.0-ce).

The used gcc docker image is: sha256:95d81930694ca9e705b71dc141ddd13f466f4989857f74aebaf1d29ba6553775

Appareantly this question is linked: Why does the order in which libraries are linked sometimes cause errors in GCC?

Now my question: When both systems run a docker container. Why does changing the linking order in this case fix the problem?

score 2 · Answer 1 · answered Jan 08 '18 at 15:42

2

Dependencies. Order of files matters.

The linker processes the static libraries one at a time, and resolves the missing symbols and pulls those into the executable file that is being created.

So if a static library (*.a) has dependencies on another static library, it has to appear before the static library that fulfills its missing symbols.

Object files (*.o) are consumed "wholesale", so order for them is less of a bother.

answered Jan 08 '18 at 15:42

Eljay

4,648
3
16
27

Okay, I am now just wondering why this is occuring in one docker instance and not in the other one. – Phillipp Mevenkamp Jan 08 '18 at 16:53
An additional issue besides external resolution is some machines/CPUs have limits on addressing and the order of linking can affect where routines/data exist in memory which can make an address fix-up work/not work; this would be a link/build time error. Also undefined program behavior (mixing pointer addresses with signed/unsigned arithmetic) may cause a bug due to memory layout as well as wild pointers and other issues. This would be a run time error as per the OP. – artless noise Jan 18 '18 at 16:30

score 1 · Accepted Answer · answered Jan 09 '18 at 08:17

The proper solution to this in CMake is not to adjust the order manually, but rather to model the inter-dependencies between different targets correctly.

The exact nature of the order dependency here is toolchain dependent (on gcc, dependees have to come before dependencies on the linker command line; MSVC doesn't care; other toolchains may choose different order requirements). The only way for CMake to ensure that it generates the correct order for the given toolchain is by modelling the dependencies explicitly in CMake.

In your example you have modeled a flat list of dependencies:

target_link_libraries(${PROJECT_NAME}_test PRIVATE Threads::Threads -lglog -lpthread
    ${QT_LIBRARIES}
    ${OpenCV_LIBRARIES}
    ${PROJECT_BINARY_DIR}/googletest-build/googlemock/libgmock.a
    ${PROJECT_BINARY_DIR}/googletest-build/googlemock/gtest/libgtest.a
)

You have a single target ${PROJECT_NAME}_test that depends on a bunch of libraries. But that is actually wrong! In reality, there is a dependency from gmock to gtest which you didn't tell CMake about. You need to model this dependency explicitly in order for CMake to work correctly. Since dependencies can only be specified between targets, we need to introduce two additional targets for gtest and gmock:

add_library(gtest INTERFACE)
target_link_libraries(gtest INTERFACE ${PROJECT_BINARY_DIR}/googletest-build/googlemock/gtest/libgtest.a)
add_library(gmock INTERFACE)
target_link_libraries(gmock INTERFACE ${PROJECT_BINARY_DIR}/googletest-build/googlemock/libgmock.a)
target_link_libraries(gmock INTERFACE gtest)   # now gmock depends on gtest

target_link_libraries(${PROJECT_NAME}_test PRIVATE Threads::Threads -lglog -lpthread
    ${QT_LIBRARIES}
    ${OpenCV_LIBRARIES}
    gtest
    gmock     # order doesn't matter here;
              # you can even omit gtest completely now
)

Note the target_link_libraries call here that establishes the dependency from gmock to gtest. It is very important to always model direct dependencies between static libraries like this in CMake, otherwise you will get problems like the one you described, which can quickly grow over your head once your build exceeds a certain complexity.

As a side note, try not to hard-code library paths in your CMake files, as that again makes your build non-portable and may break completely on different toolchains.

Why does changing the linking order fix some linking errors on one system?

2 Answers2

Linked