Is it safe to use polymorphic objects from a shared library?

Question

I have an interface, and I have an implementation of that interface in a shared library, and who knows what it was compiled with.

dynamic_cast works only on polymorphic types. So I can assume it does not use RTTI, so I do disable it, and it compiles.

And since there's no standardized way of doing polymorphism, the layout of an object of a polymorphic type is unknown, so is the layout of its vtable.

Thus the question arises: can I use an object of a polymorphic type from a shared library that was compiled with a compiler different from the one I use? And, more importantly, why?

Answer 1

C++ doesn't have a standardized ABI but if your interface follows a couple of rules, you can achieve ABI compatibility between most compilers - especially on Windows. This technique is most famously used in Microsoft's COM (https://en.wikipedia.org/wiki/Component_Object_Model), but I've also seen it in Steinberg's VST Model Architecture (https://steinbergmedia.github.io/vst3_doc/base/index.html) and in the "openvr" library (https://github.com/ValveSoftware/openvr/blob/5aa6c5f0f6520c59c4dce124541ecc62604fd7a5/headers/openvr.h#L1940).

Here are the basic rules for such ABI compatible C++ interfaces:

1. Only pure virtual methods. This makes multiple inheritance relatively unproblematic. One important caveat is that only the implementation side is allowed to perform a static_cast from one interface to another (to guarantee the right this-pointer adjustment). To solve this problem, every COM interface provides the QueryInterface() method, which is a bit similar to a dynamic_cast.
2. No virtual destructors. Some compilers generate more than 1 vtable entry (see Why do I have two destructor implementations in my assembly output?). This means you must implement your own mechanism for destructing an object from a base pointer. COM and VST3 use reference counting with addRef() and release() and they have their own type of client-side smart pointers. Alternatively, you can have a simple destroy() method and store your object instance in a regular std::unique_ptr or std::shared_ptr with a custom deleter.
3. Memory management must not cross interface boundaries, i.e. you must not allocate memory on one side and deallocate on the other side, because each side might use a different runtime. The library has to provide a free function or interface method to deallocate objects. It might also allow users to pass an allocator, so the memory management stays on the client side.
4. No overloaded virtual methods. Most compilers generally put methods in the vtable in the order of their declaration, but apparently the order of overloaded methods in the vtable varies across compilers.
5. All method parameters must be primitive types or public classes with a stable object layout (preferable PODs). You must not use any classes from the C++ standard library like std::string because the implementation is not stable.
6. Once the interface is published, it must never change. You can extend existing interfaces by inheriting from them:

class IFoo { 
public:
    virtual void foo() = 0; 
};

class IFooEx : public IFoo {
public:
    virtual void bar() = 0; 
};

or add a new interface and use multiple inheritance.

That being said, as a library author you should think twice before choosing this technique, especially if you plan to add bindings for other languages. Although the vtables of such C++ interface can be translated to C structs of function pointers, the usual approaches are

a) start with a C API and provide a client-side C++ wrapper

b) use regular C++ interfaces and provide a C layer on top of it.

But with the right abstractions, COM-like C++ interfaces can be quite nice to program with.