Missing symbol error after compiling a library that references another precompiled library

Question

tl;dr: I'm trying to compile a .cpp file that references another compiled API (.so + .h files), and I get a missing symbol when I try to load it. Below is a simple example of how I created the modules, and all the steps that I made along the way. Help would be very appreciated.

NOTE: Before you decide to skip this question because you might know nothing about Cython, this is more of a linker/g++ question than a cython question, so please read to the end.

I'm trying to create a Cython module that wraps a precompiled C++ API library, and I get strange errors on importation to python. So I have made a simple example during which:

1. Compile a simple C++ library (.so + .h files)
2. Compile a simple Cython module that references it.
3. Load that module in Python to access the underlying C++ library.

First I have the following two files:

// circle.h
#ifndef CIRCLE_H
#define CIRCLE_H

class Circle {
public:
    Circle(float radius);
    float getDiameter() const;

private:
    float m_radius;
};

#endif // CIRCLE_H

//circle.cpp
#include "circle.h"

Circle::Circle(float radius) : m_radius(radius) {}

float Circle::getDiameter() const {
    return m_radius * 2.0f;
}

I compile these into build/Circle.so using the following commands:

mkdir -p build
g++ -O2 -fPIC -c circle.cpp -o build/Circle.o
g++ -shared build/Circle.o -o build/Circle.so
cp circle.h build/Circle.h

Now I turned to create my Cython module by creating the following file named example.pyx:

# distutils: language = c++

cdef extern from "build/Circle.h":
    cdef cppclass Circle:
        Circle(float radius)
        float getDiameter()

cdef class PyCircle:
    cdef Circle* c_circle

    def __cinit__(self, float radius):
        self.c_circle = new Circle(radius)

    def __dealloc__(self):
        del self.c_circle

    def getDiameter(self):
        return self.c_circle.getDiameter()

Then I ran cythonize -i example.pyx and got two files in the root directory: example.cpp (a Cython-generated source file) & example.cpython-39-x86_64-linux-gnu.so. I renamed the latter to example.so and tried to import it from a python (import example), and I get the following error:

ImportError: example.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZNK6Circle11getDiameterEv

So I looked into the aforecompiled (that's almost a word) module Circle.so using the command nm -D build/Circle.so | grep Diameter, and indeed I saw that the missing symbol resides inside that module:

0000000000001110 T _ZNK6Circle11getDiameterEv

So I figured "there must be some problem with the linker. I know! Let's compile that auto-generated .cpp source code and link to that library!". So I did the following:

1. Ran g++ -O2 -fPIC -c example.cpp -I /usr/include/python3.9 -I build/Circle.h
2. Renamed build/Circle.so to libCircle.so
3. Ran g++ -shared example.o -Lbuild -lCircle -o example.so
4. Ran import example in the python console and I got the same error.

What am I doing wrong? How come that symbol is not loaded?

Answer 1

The core problem is that Cython doesn't know about circle.cpp and the definitions within. There are a couple ways to fix this, depending on the final use case. The first two require circle.cpp to build the Cython module, while the third just needs the shared library and headers.

Directly include circle.cpp (docs)

You can tell Cython about the circle.cpp file with a cdef extern from block, which it will #include from the generated example.cpp file:

cdef extern from "circle.cpp":
    pass

Statically link with circle.cpp (docs)

Add a directive at the top of example.pyx that tells cythonize to build circle.cpp and statically link it into example.so:

# distutils: sources = circle.cpp

Dynamically link with libCircle.so (docs)

Add some directives to example.pyx, which essentially add -Lbuild -lCircle to the linker invocation:

# distutils: library_dirs = build
# distutils: libraries = Circle

You'll need to make sure libCircle.so is somewhere that the dynamic linker knows about: either install it, add the build directory to $LD_LIBRARY_PATH, or add an rpath entry to resolve it relative to example.so.

Also see this SO answer for a very detailed breakdown of how dynamic library loading works in Cython.