98

Background

In the last year I was using the nlohmann json library[1] and was cross-compiling on x86_64 using GCC 5.x arm-linux-gnueabi-* with no warnings. When I updated GCC to a newer version, GCC would generate pages of cryptic diagnostic notes. For example, here is one of the notes

In file included from /usr/arm-linux-gnueabi/include/c++/7/vector:69:0,
             from include/json.hpp:58,
             from src/write_hsi.cpp:23:
/usr/arm-linux-gnueabi/include/c++/7/bits/vector.tcc: In member function ‘void std::vector<_Tp, _Alloc>::_M_realloc_insert(std::vector<_Tp, _Alloc>::iterator, _Args&& ...) [with _Args = {nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long int, long long unsigned int, double, std::allocator, nlohmann::adl_serializer>}; _Tp = nlohmann::basic_json<>; _Alloc = std::allocator<nlohmann::basic_json<> >]’:
/usr/arm-linux-gnueabi/include/c++/7/bits/vector.tcc:394:7: note: parameter passing for argument of type ‘std::vector<nlohmann::basic_json<>, std::allocator<nlohmann::basic_json<> > >::iterator {aka __gnu_cxx::__normal_iterator<nlohmann::basic_json<>*, std::vector<nlohmann::basic_json<>, std::allocator<nlohmann::basic_json<> > > >}’ changed in GCC 7.1
   vector<_Tp, _Alloc>::
   ^~~~~~~~~~~~~~~~~~~
/usr/arm-linux-gnueabi/include/c++/7/bits/vector.tcc: In member function ‘nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer> nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::parser::parse_internal(bool) [with ObjectType = std::map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long long int; NumberUnsignedType = long long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer]’:
/usr/arm-linux-gnueabi/include/c++/7/bits/vector.tcc:105:21: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<nlohmann::basic_json<>*, std::vector<nlohmann::basic_json<>, std::allocator<nlohmann::basic_json<> > > >’ changed in GCC 7.1
_M_realloc_insert(end(), std::forward<_Args>(__args)...);
~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It was easy to find a solution, namely by adding -Wno-psabi to the compiler options. In fact, that was the fix implemented in the library.[2]

I understand the basics of Application Binary Interfaces (ABIs) and processor-specific ABIs (psABIs). For reference, this answer[11] gives a quick overview of ABIs:

An ABI (Application Binary Interface) is a standard that defines a mapping between low-level concepts in high-level languages and the abilities of a specific hardware/OS platform's machine code. That includes things like:

  • how C/C++/Fortran/... data types are laid out in memory (data sizes / alignments)
  • how nested function calls work (where and how the information on how to return to a function's caller is stored, where in the CPU registers and/or in memory function arguments are passed)
  • how program startup / initialization works (what data format an "executable" has, how the code / data is loaded from there, how DLLs work ...)

The answers to these are:

  • language-specific (hence you've got a C ABI, C++ ABI, Fortran ABI, Pascal ABI, ... even the Java bytecode spec, although targeting a "virtual" processor instead of real hardware, is an ABI),
  • operating-system specific (MS Windows and Linux on the same hardware use a different ABI),
  • hardware/CPU-specific (the ARM and x86 ABIs are different).
  • evolving over (long) time (existing ABIs have often been updated / rev'ed so that new CPU features could be made use of, like, say, specifying how the x86 SSE registers are to be used by apps was of course only possible once CPUs had these regs, therefore existing ABIs needed to be clarified).

So the ABI is the overarching component, and one of its components (the "hardware/CPU-specific" details) is the psABI.

My Issue

The problem I am having is

  1. I don't like universally disabling warnings without understanding the implications.
  2. The advice "use -Wno-psabi to make the notes go away" seems to be pretty common advice for these types of diagnostic notes that "suddenly appear" after a compiler upgrade.[2][3][4] Even the one of the GCC developers suggest doing this.[5]
  3. Neither -Wpsabi nor -Wno-psabi are documented[6] in the GCC manual.[7]

As a result I am not really sure what exactly -Wno-psabi will and will not affect. A related option -Wabi is documented:[8]

-Wabi (C, Objective-C, C++ and Objective-C++ only)

Warn when G++ it generates code that is probably not compatible with the vendor-neutral C++ ABI...

It also warns about psABI-related changes. The known psABI changes at this point include:

  • For SysV/x86-64, unions with long double members are passed in memory as specified in psABI. For example:

union U { long double ld; int i; };

union U is always passed in memory.

My understanding of this all is

  1. -Wabi will generate warnings when there is a psABI change.
  2. GCC 7 fixed an ABI bug[9] introduced in GCC 5 that affects ARM targets.
    • In the release notes it is stated "this is an ABI change."[10]
    • For some reason the release notes state that the related diagnostic notes are generated when using the undocumented -Wpsabi, not the documented -Wabi.
    • This ABI change is not mentioned in the manual.
  3. Putting together "this is an ABI change" and "use -Wpsabi", it appears to me this is specifically a psABI change, not a different kind of ABI change. (In reality it is a change in GCC's implementation of the psABI, not the psABI itself)

I know that documentation isn't always up to date, especially for something that is a known undocumented option. But my concern is that "use -Wno-psabi" seems to be the standard response for several different kinds of these cryptic diagnostic notes. But, in my basic understanding of ABIs, isn't an ABI change a big deal? Shouldn't I be concerned about an ABI change, rather than just making the message go away? Between the undocumented stuff and some of the finer details of ABI vs psABI, I'm not really sure...

For example, if I add -Wno-psabi to my makefile to make those notes go away, what if there is another ABI change in the future that does affect my project? Have I effectively silenced future warnings or notes that may be important?

Also, even though we are told "if you recompile all the code, there is nothing to worry about,"[5] what exactly is "all the code"? Is that my source code? glibc? Any other system-wide shared library I might be using?

References

  1. https://github.com/nlohmann/json
  2. https://github.com/nlohmann/json/issues/658
  3. https://stackoverflow.com/a/48149400
  4. https://stackoverflow.com/a/13915796/10270632
  5. https://gcc.gnu.org/ml/gcc/2017-05/msg00073.html
  6. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81831
  7. https://gcc.gnu.org/onlinedocs/gcc-8.2.0/gcc
  8. https://gcc.gnu.org/onlinedocs/gcc-8.2.0/gcc/C_002b_002b-Dialect-Options.html
  9. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77728
  10. https://gcc.gnu.org/gcc-7/changes.html
  11. https://stackoverflow.com/a/8063350
rmc
  • 1,118
  • 1
  • 7
  • 9
  • @jesper-juhl Thanks. I *scoured* SO, mailing lists, etc. looking for an answer. I even spent a bit of time in the GCC source code trying to figure it out (and maybe be able to contribute a patch to fix [the missing documentation](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81831)), but I wasn't familiar enough with GCC internals to figure it out. It feels like using a sledgehammer to hammer a nail, and then forgetting that normal hammers exist... – rmc Aug 25 '18 at 19:31
  • I'm just sad I don't have a good answer for you. But hopefully someone will drop by who does. :) – Jesper Juhl Aug 25 '18 at 19:35
  • @jesper-juhl That makes two of us :) I guess it was inevitable that some day I would have a question that didn't already have a SO answer and would be forced to make an account to ask about it. – rmc Aug 25 '18 at 19:48
  • I don't think this will affect you. If the standard library is compiled with the same GCC version (I mean a version which uses the same abi) that you use (which is likely), everything should be fine. – geza Aug 25 '18 at 20:16
  • @geza I don't think the standard library is from the same version of GCC. Those messages appeared when I updated the cross-compiler on the host x86_64 computer. The ARM target machine has GCC 5.x on it. I guess that would leave me the choice between either reinstall the GCC 5.x cross compiler, compile on target, statically link *everything*, or figure out a way to update the packages on the non-networked target machine. Also, even if that's a non-issue, I still would like to figure out all of the different things `-Wpsabi` affects (the first part of the question). – rmc Aug 25 '18 at 20:57
  • [AMD64 **P**rocessor **S**upplement ABI](https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnx4MzJhYml8Z3g6MzVkYzQwOWIwOGU4YzViYw) - the Introduction chapter is instructive. My _guess_ would be that ABI changes or PS ABI changes are treated the same, but have two different compiler flags for supressions. The docs for `-Wabi` should apply also for `-Wpsabi`/`-Wno-psabi`, where the latter refers to the changes due to conformance to the PS ABI rather than the ABI. – dfrib Aug 25 '18 at 21:06
  • @rmc: Ah. Somehow I supposed that you're targeting android. Note, it is likely that the warnings you quoted don't do any harm. It is template code, which gets included into the elf. So, there would be no ABI mismatch between your code and the compiled libstdc++. But of course, you'll need to check out all the warnings, because maybe there is something in there, which can be potentially bad (you need to recompile nlohmann json library with the new compiler, of course). – geza Aug 25 '18 at 21:13
  • Citing [System V ABI](https://wiki.osdev.org/System_V_ABI): _"It is today the standard ABI used by the major Unix operating systems ... The ABI is organized as a portable base document **and platform-specific supplements** that fill in the blank gaps ... Unofficial new architecture processor supplements have been published as the format has been adapted to new platforms ... Due to the many unofficial supplement specifications and the chaotic history of the Unix operating systems ... The ABI is well-understood by common operating system development tools like Binutils and GCC"_. – dfrib Aug 25 '18 at 21:20
  • You are using the nlohmann only via header, not as linkable library archive or shared-object? – Peter Aug 27 '18 at 06:51
  • It is possibly related to GCC bug fixes. Here is a new instance: https://gcc.gnu.org/gcc-9/changes.html . – FrankHB Mar 16 '20 at 07:38
  • 1
    @Peter, yes we were using nlohmann as header-only – rmc Mar 08 '21 at 16:34
  • @FrankHB I think that actually illustrates exactly the kind of situation I was worried about if I disabled `-Wpsabi`. The ABI change warning I experienced occurred after updating from GCC 5.x to a newer version (I can't remember if it was a newer 5.x or 6.x). The GCC 9 changes you linked are for a bug "On Arm targets... in the GCC 6, 7 and 8 releases... This is an ABI change. If the option -Wpsabi is enabled... the compiler will emit a diagnostic note..." If I had put -Wno-psabi in my makefile 2 years ago while using GCC 5.x, it would have suppressed warnings on this later ABI change. – rmc Mar 08 '21 at 16:41

1 Answers1

15

You only need to worry about ABIs when you are crossing library boundaries. Within your own applications/libraries the ABI doesn't really matter as presumably all your object files are compiled with the same compiler version and switches.

If you have a library compiled with ABI1 and an application compiled with ABI2 then the application will crash when it tries to call functions from the library as it wont pass the arguments correctly. To fix the crash you would need to recompile the library (and any other libraries it depends on) with ABI2.

In your specific case as long as you compile nlohmann with the same compiler version as your application (or are just using nlohmann as a header) then you don't need to worry about the ABI change.

Globally suppressing the warning seems to be a dangerous option as it will prevent you from seeing any future ABI issues. A better option would be to use #pragma to disable the warning just for the functions in question, e.g.:

#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wno-psabi"
void foo()
{
}
#pragma GCC diagnostic pop
Alan Birtles
  • 32,622
  • 4
  • 31
  • 60
  • Sorry to bring back to life a very old post but you say that when calling a library compiled in ABI1 from an application compiled with ABI2 the application will crash, is this guaranteed behaviour or are you using the term "crash" as a general term to describe "application will misbehave". Taking the specific example of Redhat, there they just say it is not supported and not guaranteed to work, not that it will crash. To be honest I'd much rather a crash, but would like to understand where this statement came from. – hhafez Nov 24 '20 at 03:06
  • 4
    Yes, no guarantee it'll crash, the behaviour will be undefined – Alan Birtles Nov 24 '20 at 07:12
  • undefined is a dangerous zone, it can give the unwary a false sense of security when things "look okay" then when in it all falls apart. Much rather the application fails to start and the problem gets sorted – hhafez Nov 25 '20 at 00:36
  • @alan-birtles Wow, I haven't logged on in years and forgot I even asked this question. I'm going to mark your answer as accepted, as I think it hits all the points I had been wondering about. IIRC, since this was for an embedded target and we had sources for all 3rd party libraries, we ended up just recompiling everything with the updated cross-compiler. – rmc Mar 08 '21 at 16:33
  • 4
    "-Wno-psabi" isn't recognised by the gcc diagnostic pragma for me. Code won't build with that line in it. – fuzzyTew May 05 '21 at 06:33