5

I have a Go library that provides bindings for the C++ OpenImageIO library (OpenImageiGO). I've been happily building my bindings via the standard dynamic linking to libOpenImageIO, but am now trying to link statically. I'm running into an issue where no matter which combination of flags I try, the external linker fails with a ton of "undefined reference" errors. I seem to recall seeing this issue raised in the past, saying that there was an issue regarding the order in which the linker saw the symbols. But I can't seem to find this information again.

Here is a brief example of my most recent build attempt, trying to get it to link against static builds of boost, OpenColorIO, and OpenImageIO :

$ export CGO_CPPFLAGS="\
-I/path/to/boost/include \
-I/path/to/OpenColorIO/include \
-I/path/to/OpenImageIO/include"

$ export CGO_LDFLAGS="\
-L/path/to/boost/lib -lboost_thread_static -lboost_system_static \
-L/path/to/OpenColorIO/lib -lopencolorio \
-L/path/to/OpenImageIO/lib -lOpenImageIO"

$ go build -v -x --ldflags '-extldflags "-static"'  github.com/justinfx/openimageigo
...
CGO_LDFLAGS="/path/to/boost/lib/libboost_system_static.a" "/path/to/boost/lib/libboost_thread_static.a" "/path/to/OpenColorIO/lib/libopencolorio.a" "/path/to/OpenImageIO/lib/libOpenImageIO.a" "-lstdc++" /vol/apps/go/1.3.0/pkg/tool/linux_amd64/cgo -objdir $WORK/github.com/justinfx/openimageigo/_obj/ -- -I/path/to/boost/include -I/path/to/OpenColorIO/include -I/path/to/OpenImageIO/include -I./cpp -I $WORK/github.com/justinfx/openimageigo/_obj/ -I/path/to/boost/include -I/path/to/OpenColorIO/include -I/path/to/OpenImageIO/include color.go imagebuf.go imagebufalgo.go imagecache.go imageinput.go imageoutput.go imagespec.go oiio.go roi.go
...
/usr/bin/g++ -I . -fPIC -m64 -pthread -fmessage-length=0 -I/path/to/boost/include -I/path/to/OpenColorIO/include -I/path/to/OpenImageIO/include -I./cpp -I $WORK/github.com/justinfx/openimageigo/_obj/ -g -O2 -o $WORK/github.com/justinfx/openimageigo/_obj/all.cpp.o -c ./all.cpp

/usr/bin/g++ -I . -fPIC -m64 -pthread -fmessage-length=0 -o $WORK/github.com/justinfx/openimageigo/_obj/_cgo_.o $WORK/github.com/justinfx/openimageigo/_obj/_cgo_main.o $WORK/github.com/justinfx/openimageigo/_obj/_cgo_export.o $WORK/github.com/justinfx/openimageigo/_obj/color.cgo2.o $WORK/github.com/justinfx/openimageigo/_obj/imagebuf.cgo2.o $WORK/github.com/justinfx/openimageigo/_obj/imagebufalgo.cgo2.o $WORK/github.com/justinfx/openimageigo/_obj/imagecache.cgo2.o $WORK/github.com/justinfx/openimageigo/_obj/imageinput.cgo2.o $WORK/github.com/justinfx/openimageigo/_obj/imageoutput.cgo2.o $WORK/github.com/justinfx/openimageigo/_obj/imagespec.cgo2.o $WORK/github.com/justinfx/openimageigo/_obj/oiio.cgo2.o $WORK/github.com/justinfx/openimageigo/_obj/roi.cgo2.o $WORK/github.com/justinfx/openimageigo/_obj/all.cpp.o /path/to/boost/lib/libboost_system_static.a /path/to/boost/lib/libboost_thread_static.a /path/to/OpenColorIO/lib/libopencolorio.a /path/to/OpenImageIO/lib/libOpenImageIO.a -lstdc++

And here are some of the cherry-picked errors, since it was a very long bit of output:

/path/to/OpenImageIO/lib/libOpenImageIO.a(OpenImageIO_dist^src^libOpenImageIO^color_ocio.cpp.o): In function `ColorConfig':
/path/to/OpenImageIO/OpenImageIO_dist/src/libOpenImageIO/color_ocio.cpp:141: undefined reference to `OpenColorIO::v1::SetLoggingLevel(OpenColorIO::v1::LoggingLevel)'
...
/path/to/OpenImageIO/lib/libOpenImageIO.a(OpenImageIO_dist^src^libOpenImageIO^imagebufalgo_copy.cpp.o): In function `boost::shared_mutex::lock()':
/path/to/boost/include/boost/thread/pthread/shared_mutex.hpp:138: undefined reference to `boost::this_thread::disable_interruption::~disable_interruption()'

OpenImageIO can't seem to find references for OpenColorIO. And, OpenImageIO can't seem to find references for boost. It appears like the order in which things are happening during the linking isn't making the OpenColorIO or boost symbols available to OpenImageIO, so I get a bunch of symbol errors.

I'm hoping I am doing something simple and stupid that can be corrected in my build process. But the cgo static linking with external libs does seem a touch more complicated than the default dynamic linking approach.

Update #1

The answer given by @james-henstridge was right on, and I am almost fully built, except for one last hiccup. I'm getting failing references for yaml-cpp, needed by OpenColorIO, even though it seems I have the correct order.

Here is my latest env, where I have worked through all of the explicit static libs that had to be added:

$ export CGO_CPPFLAGS="-I/usr/local/include -I/usr/include"

$ export CGO_LDFLAGS="\
-L/usr/local/lib \
-L/usr/lib \
-L/usr/lib/x86_64-linux-gnu \
-lOpenImageIO \
-lHalf -lIex -lfreetype -lIlmThread -lImath -lIlmImf -lIlmThread \
-lOpenColorIO \
-lyaml-cpp -ltinyxml \
-lboost_regex -lboost_filesystem -lboost_thread -lboost_system \
-ltiff -lgif -lpng -ljpeg -lz \
-lrt -ldl"

$ go test -v -x --ldflags '-extldflags "-static"' github.com/justinfx/openimageigo
...
/home/justin/src/OpenColorIO/src/core/OCIOYaml.cpp:329: undefined reference to `YAML::Node::begin() const'
...
/home/justin/src/OpenColorIO/build/ext/dist/include/yaml-cpp/nodereadimpl.h:79: undefined reference to `YAML::Node::GetScalar(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&) const'
...
/usr/local/lib/libOpenColorIO.a(OCIOYaml.cpp.o): In function `_FindFromNodeAtIndex':
/home/justin/src/OpenColorIO/build/ext/dist/include/yaml-cpp/nodeutil.h:53: undefined reference to `YAML::Node::FindAtIndex(unsigned long) const'
collect2: ld returned 1 exit status

Update #2

Nevermind about update #1. It was specifically related to OpenColorIO and not a general issue.

jdi
  • 90,542
  • 19
  • 167
  • 203

1 Answers1

7

The order of the -l flags matters when you are linking with static libraries. If you link using -lfoo -lbar -lbaz, any symbols required by libbar.a will only be searched for in libbar.a and libbaz.a. Even if libfoo.a contains the symbols you're after, the linker won't find them.

What is happening is that for each library, the linker unpacks the archive and adds the object files that contain symbols referenced by whatever came before. If a particular object file in the archive isn't needed, it is ignored.

The fix is to make sure every library is listed before any it depends on in the linker flags. If there are any dependency loops (which there shouldn't be), it may be necessary to list a library twice.

James Henstridge
  • 42,244
  • 6
  • 132
  • 114
  • 1
    Ooo so if I had listed OpenImageIO first, then OpenColorIO (which is needed by the former), and then boost (which is needed by the others), this might have compiled? Will test and report back! – jdi Jul 12 '14 at 05:13
  • Thanks a lot. This seems to be the issue. If you don't mind taking one last look at my update to the original question? It seems to still be giving me trouble with *one last* reference before linking successfully, but this one is a bit difference since I think I have the right order here. – jdi Jul 13 '14 at 09:10
  • How was libyaml-cpp.a built? Was `ranlib` run on the resulting archive as part of the build process? Without that, you can run into problems resolving symbol dependencies within the archive. – James Henstridge Jul 13 '14 at 12:47
  • 1
    You know what? Probably disregard my problem with `yaml-cpp`. Apparently `OpenColorIO` (dependency) has some kind of unexported setup with `yaml-cpp` and `tinyxml`. I get the same problem at work that I do at home. Most likely related to this: https://github.com/imageworks/OpenColorIO/issues/318 – jdi Jul 14 '14 at 01:33
  • Oldie but goodie. This holds true for Visual Studio libs as well. – soulsabr Jun 06 '19 at 20:27