0

I am trying to expose some c++ code to Python with pybind11. I specifically would like to enforce a certain c++ standard (say c++11), as the same .cpp file would need to be compiled on different systems. Following the official example to compile with setuptools in this repository, I modified part of the code as follows:

ext_modules = [
    Pybind11Extension(
        modulename,
        sources=[filename],
        language='c++',
        cxx_std=11
    ),
]

I ran python setup.py install on my windows machine and a Linux machine (it's actually Google Colab, which uses Linux backend). I checked the standard used by adding py::print(__cplusplus); in code. On the Linux machine, the standard changed from c++14 to c++ 11 after adding cxx_std=11. However, on my own windows machine, the standard remained as c++98. In both cases, the exposed code can be sucefully imported as a module (a .pyd file on windows and a .so file on linux) and works as intended. This is very strange because:

(1) According to the documentation for Pybind11Extension, it "Build a C++11+ Extension module with pybind11.". It does not make sence for my code to be compiled with c++98 successfully when the minimum requirement is c++11.

(2) I have gcc version 8.1.0 (by checking g++ -v), which should support the latest standard of c++. Additionally, if I use g++ command to compile a normal cpp file, it says c++14, which means c++14 is the default.

This difference of standard is causing a lot of trouble because sometimes code works on my windows machine but fails when compiled on Linux.

Now, I am relatively new to stuff about c++ compilers, so I might be missing something obvious. Any idea how to solve this?

Update: After some discussion it seems that setup.py uses MSVC on windows. This question then becomes a duplicate of this, which remains unsolved (the setup.cfg method mentioned does not work. I could specify --compiler msvc but if I write --compiler mingw32 it raise several errors: gcc: error: /EHsc: No such file or directory, gcc: error: /bigobj: No such file or directory, gcc: error: /std:c++14: No such file or directory).

Update 2: With the method by spectras it is confirmed that setup.py uses MSVC, which also has c++14 as earlist possible standard. However, I still want to enforce GCC so I could have the same compiler on windows and linux. To enforce GCC, I tried the following command (env_csmar is an anaconda virtual environment under my disk F):

D:\mingw-w64\x86_64-8.1.0-posix-seh-rt_v6-rev0\mingw64\bin\gcc.exe -static -shared -std=c++11 -DMS_WIN64 -fPIC "-IF:\...\env_csmar\lib\site-packages\pybind11\include" "-IF:\...\env_csmar\include" "-L F:\...\env_csmar\libs" -c example_cpp.cpp -o example_cpp.o

This works, and the .o file can be imported as a module on my windows. However, it still says it is compiled with MSVC and under c++14 (from #ifdef _MSVC_LANG). Why is this?

Jerry
  • 107
  • 7
  • C++98 compliant code should also compile fine as C++11 code. The standard is backwards compatible. Therefore it's more likely that your C++ code uses non-standard extensions that don't work on Linux – UnholySheep Jun 08 '22 at 14:48
  • @UnholySheep I had one base class and one derived class, both with a template type. (`template class DerivedClass : public BaseClass {...};`). The derived class with `D` as `double` is exposed to Python. Then in one of the virtual functions of the derived class, the code tried to fetch the value of a protected member variable of the base class. This line of code throws an error of something like this variable is not defined when compiled on Linux, but compiles and runs fine on windows. Is this non-strandard extension? – Jerry Jun 08 '22 at 14:56
  • MSVC is known to be a lot more lax when it comes to requirements for templates, so it is very much possible you ran into such a case (we'd need a full [mre] to be able to tell for sure) – UnholySheep Jun 08 '22 at 15:18
  • @UnholySheep yeah I actually did a minimal reproducible example but then the point of the post is not about this so I didn't post it. Can you explain a bit about MSVC? I thought it used g++ for compilation. – Jerry Jun 08 '22 at 16:04
  • @UnholySheep I did a bit of research, so basically it is very likely that it compiled with MSVC on windows, and failed on the more conforming GCC on linux? – Jerry Jun 08 '22 at 16:19
  • Yes, MSVC is the usual compiler used on Windows (there's also versions of GCC and Clang that can be used on Windows) and it would explain the problem you have. – UnholySheep Jun 08 '22 at 16:24
  • 1
    I think MSVC also always reports the C++98 value for `__cplusplus` without an explicit compiler option set, even if it really is set to a different standard revision. So you can't rely on that. – user17732522 Jun 09 '22 at 03:41

1 Answers1

1

You did not build using C++98 on Windows.

You got confused because by default MSVC always reports __cplusplus as 199711, no matter what standard it is using, for compatibility reasons.

This behavior can be disabled with the /Zc:__cplusplus switch.

Alternatively, you can use _MSVC_LANG predefined macro. If it is defined, then you are building with MSVC (or a compiler emulating it) and then _MSVC_LANG will accurately reflect the language standard being enforced.

Something like…

#ifdef _MSVC_LANG
py::print(_MSVC_LANG)
#else
py::print(__cplusplus)
#endif

…should do the trick.

spectras
  • 13,105
  • 2
  • 31
  • 53
  • This explains half of the problem! Now it is showing 201402, but I still haven't found a way to enforce GCC or c++11 with MSVC. – Jerry Jun 09 '22 at 04:07
  • Alas I don't know pybind and its interaction with setup.py, but at least you have reliable feedback from your code now. For what it's worth, MSVC accepts a `/std:c++14`, `/std:c++17`, `/std:c++20`, … Do note the earliest is c++14, there is no support for earlier versions of the language. – spectras Jun 09 '22 at 04:20
  • yeah I changed to `cxx_std=17` and it works, than you so much! I guess the only problem now is how to enforce GCC. – Jerry Jun 09 '22 at 04:29