1

There are many articles on the internet that say that Python cannot access C++ compiled DLLs using ctypes, and there are other contradictory articles which say that it is possible to access the DLL so long as the DLL is compiled with the extern "C" method. So I just want clarification. Is it possible if extern "C" is used ?

Robert Tattorn
  • 107
  • 5
  • 3
    Yes, of course. They look the same as C DLLs – user253751 Sep 01 '20 at 13:35
  • 1
    I have done exactly this, on a LINUX platform, using this info: https://nesi.github.io/perf-training/python-scatter/ctypes. I have never tried it on a Windows platform. – NateB Sep 01 '20 at 13:38
  • 1
    *Python cannot access C++ compiled DLLs using ctypes* -- This is not true, as the open source product I maintain (a C++ compiled DLL) is easily accessible using python. – PaulMcKenzie Sep 01 '20 at 13:39
  • Sure, for example, this answer: https://stackoverflow.com/a/62528026/235698 – Mark Tolonen Sep 01 '20 at 16:12

1 Answers1

3

There are C++-compiled dlls which can be used with ctypes and C++-compiled dlls which cannot be used with ctypes directly.

If API exposes only plain-old-data (e.g. int, int *, char * and similar) and is wrapped in extern "C" then it can be consumed from ctypes.

If non-POD-classes are used (for example std::vector or std::string) in interfaces or the API isn't wrapped in extern "C" then the dll cannot be used with ctypes (at least in a portable way and without much ado).

Why extern "C" is needed?

Following declarations are valid in C++ but not in C:

void my_fun(int a);
void my_fun(double a);

Because C doesn't perform name mangling, both functions would be mapped to a symbol named my_fun in the resulting object file, which is problematic.

C++ would create two different symbols with mangled names, for example gcc would create symbols Z6my_funi and _Z6my_fund. MSVC has another naming scheme - so the resulting symbols in the dll (or shared object) depends on the compiler with which it was built.

It is not impossible to find these symbols in the dll with help of ctypes - it is only not as straight forward as with C-names because one needs additional information - with which compiler the dll was build.

C-names work out of the box with ctypes, and wrapping declarations into extern "C" switches the name mangling off, thus

extern "C" {
  void my_fun(int a);
  void my_fun(double a);
}

would no longer compile, as compiler will ensure that there are no multiple definitions of a symbol.

Why more complicated C++-classes cannot be used in interfaces?

extern "C" doesn't forbid usage of std::vector<> and other C++-classes:

extern "C" {
  void my_fun(std::vector<double> a);
}

compiles, it only produces symbol my_func instead of ?my_fun@@YANV?$vector@NV?$allocator@N@std@@@std@@@Z (with MSVC) or _Z6my_funSt6vectorIdSaIdEE (with gcc) in the resulting object file.

While for PODs, let's say double[10], the memory layout is clear, the memory layout of e.g. a std::vector is implementation dependent and is not known to ctypes. Another issue are - something ctypes doesn't handle as well - constructors/destructors which can be more complicated than simple initialization of PODs.

Thus the functionality of my_fun(std::vector<double>) cannot be used out-of-the-box with ctypes. However, if one knows the memory layout of the classes, one could emulate them with ctypes, initialize objects by calling (right) constructors (btw. symbols of class-methods are always mangled, even if wrapped in extern "C"), call the function my_fun and then destroy objects by calling (right) destructors.

ead
  • 32,758
  • 6
  • 90
  • 153