Does reinterpret_cast of an int64_t value really break strict aliasing?

Question

I'm attempting to write a generic version of __builtin_clz that handles all integer types, including signed ones. To ensure that conversion of signed to unsigned types doesn't change the bit representation, I decided to use reinterpret_cast.

I've got stuck on int64_t which unlike the other types doesn't seem to work with reinterpret_cast. I would think the code below is correct but it generates a warning in GCC.

#include <cstdint>

int countLeadingZeros(const std::int64_t value)
{
    static_assert(sizeof(std::int64_t) == sizeof(unsigned long long));
    return __builtin_clzll(reinterpret_cast<const unsigned long long&>(value));
}

(demo)

GCC shows a warning: dereferencing type-punned pointer will break strict-aliasing rules.

Clang compiles it without a complaint.

Which compiler is right? If it is GCC, what is the reason for the violation of strict-aliasing?

Edit: After reading the answers, I can see that the described behavior applies not only to conversion int64_t -> unsigned long long but also to long -> long long. The latter one makes the problem a little more obvious.

@nickie Do you mean the `&` in the type passed to `reinterpret_cast`? Because it is mandatory: https://stackoverflow.com/a/2206177/3052438 — Piotr Siupa, Oct 05 '22 at 22:20
You should just use `static_cast` which converts between signed and unsigned w/o UB ever since c++17 which requires signed is two's compl. — doug, Oct 05 '22 at 22:26
@doug Does the standard guarantees the cast won't drop the minus sign bit or change the bit representation in another way? If so, I'll use `static_cast`. Still my question stands, even if only to report a bug in a compiler. — Piotr Siupa, Oct 05 '22 at 22:31
You might want to make sure `static_assert(std::is_same::value);` for both gcc and clang. Also, these kinds of warnings are not required by the standard, and what compilers provide for some kinds of warnings may only detect a subset of the violations. — Eljay, Oct 05 '22 at 22:41
@Eljay Both compilers return `false` but I have no idea what are the implications of that. (https://godbolt.org/z/ef5oajrsa) — Piotr Siupa, Oct 05 '22 at 22:49
Yes, the c++ standard as of c++17 guarantees interconvertability between signed and unsigned of the same size. It is still UB if one overflows signed ops but that's not an issue here. — doug, Oct 05 '22 at 22:59
I dont think you need any cast around the types. https://godbolt.org/z/9eEPxnbrP — Something Something, Oct 05 '22 at 23:28

score 3 · Accepted Answer · answered Oct 07 '22 at 22:52

3

If you have a signed integer type T, you can access its value through a pointer/reference to the unsigned version of T and vice-versa.

What you cannot do is access its value through a pointer/reference to the unsigned version of U, where U is not the original type. That's undefined behavior.

long and long long are not the same type, no matter what the size of those types say. int64_t may be an alias for a long, a long long, or some other type. But unless you know that int64_t is an alias for signed long long (and no, testing its size is not good enough), you cannot access its value through a reference to unsigned long long.

answered Oct 07 '22 at 22:52

Nicol Bolas

449,505
63
781
982

Can `long` and `long long` be implemented differently if they have the same size or this is just some limitation designed to make C++ harder to use? Comments under the question suggest C++ is required to use two's complement for signed integer so I don't see a lot of wiggling room for the implementation. – Piotr Siupa Oct 08 '22 at 08:13
1

@PiotrSiupa: "*Can long and long long be implemented differently*" Irrelevant. GCC is warning you about undefined behavior. That's a matter of standard-definitions, not whether you might be able to get away with it or not. The standard says that `long` and `long long` are different types. Therefore, `signed long` and `unsigned long long` are not signed/unsigned versions of the same type. Therefore, accessing one through a pointer/reference to the other is UB. – Nicol Bolas Oct 08 '22 at 13:23
1

@NicolBolas: When the Standard uses the term "Undefined Behavior", all that means as far as the Standard is concerned is that *the Standard imposes no requirements*. It's important to know that some compilers interpret that as an invitation to behave nonsensically even though other compilers would define the behavior, but that doesn't mean that compilers that behave nonsensically shouldn't be recognized as outliers. – supercat Oct 08 '22 at 14:42
@supercat: "*compilers that behave nonsensically shouldn't be recognized as outliers*" I'm not sure what statistical methodology leads one to the conclusion that 2 of the most widely used compilers are "outliers". "*the Standard imposes no requirements*" Yes, and this question exists because the compiler recognized that and reported a warning. The question is about whether the behavior is undefined or not. – Nicol Bolas Oct 08 '22 at 15:03
@NicolBolas: They are popular because they are freely distributable. Their behavior in many cases contradicts that of all predecessors that targeted commonplace octet-based blatforms, and the maintainers of each compiler frequently point to the behavior of the other as justification for their own abberent behaviors. For example, even though the Standard specified that programmers may indicate reliance upon decades-old Common Initial Sequence guarantees by ensuring that a *complete union type declaration* containing the involved structure types will be visible at the point of reliance... – supercat Oct 08 '22 at 17:02
...both clang and gcc are willffully blind to the presence of such declarations in scope when processing expressions that don't use lvalues of the union types, and point to each others' treatment of such constructs as justification for ignoring rules of type visibility. As such, I think it is entirely fair to partition the universe of C compilers into "clang and gcc" and "other compilers not based on clang and gcc", and recognize that programer's willingness to target inferior compilers which are freely distributable over quality compilers that aren't doesn't mean the latter aren't inferior. – supercat Oct 08 '22 at 17:07
@supercat: "outlier" is a statistical term with a specific meaning, not a specifier of quality. You can choose to believe that these compilers are not "good" by some definition. But claiming that they are "outliers" when they are in fact 2 of the 3 dominant compilers in the industry used by literally millions of programmers the world over is simply factually untrue. – Nicol Bolas Oct 08 '22 at 17:14
@NicolBolas: Besides, I think it's fair to say that the Standard is intended to describe the behavior of correct compilers, and so far as I am aware all compiler configurations that correctly handle all scenarios where allocated storage is written as type A, read as A, written as B, read as B, written as A, and read as A. also handle constructs the clang/gcc maintainers insist are erroneous. If you know of any exceptions, tell me. – supercat Oct 08 '22 at 17:17
@NicolBolas: Simply put, the language dialects that the clang and gcc optimizers seek to process has diverged from the language dialects the Standards were commissioned to describe, and that are processed by compilers not based on clang and gcc. When the Standards say implementations may process code "in a documented manner characteristic of the environment", that wasn't just a theoretical possibility, but in many cases that was the way compilers behaved and were expected to continue behaving, when targeting environments where that would be useful. – supercat Oct 08 '22 at 17:27
@NicolBolas: Besides, the first and second definition of outlier at dictionary.com (ahead of the statistical definition, which is third) are "something that lies outside the main body or group that it is a part of, as a cow far from the rest of the herd, or a distant island belonging to a cluster of islands" and "someone who stands apart from others of his or her group, as by differing behavior, beliefs, or religious practices." Do you disagree with dictionary.com? – supercat Oct 08 '22 at 20:23
@supercat: I guess that depends on how you define "group". You seem to think that Joe-Random-C-Compiler is just as much a part of the group as GCC, despite the userbase of GCC dwarfing some random compiler. To me, the only group that matters is the group of compiler *users*, and users of GCC and Clang are the majority. That makes "the group" GCC and Clang. – Nicol Bolas Oct 08 '22 at 21:25
1

It may be worth adding that _`long` and `long long` are not the same type_ for **type-based alias analysis** regardless of their (matching) size and alignment. – Maxim Egorushkin Oct 08 '22 at 22:24
@MaximEgorushkin: Isn't that what I said when I said that "`long` and `long long` are not the same type, no matter what the size of those types say?" – Nicol Bolas Oct 08 '22 at 22:29
That didn't occur to me prompting me to comment. They can be layout-compatible types, but always different types for type-based alias analysis. – Maxim Egorushkin Oct 08 '22 at 22:41
@NicolBolas: Does everyone in that group use the type-based aliasing optimizations, or do wise people turn them off? What fraction of the people who use clang and gcc's optimizations settings are aware of all the sitautions where the Standard was written to allow compilers to deviate from commonplace behavior in situations where doing so would allow them to be more useful, and where clang or gcc only process code meaningfully because they haven't yet found "optimization" opportunities? How many people, given a choice between having `uint1 = ushort1 * ushort2;` behave in the manner... – supercat Oct 09 '22 at 06:11
...described in the Rationale the way implementations for commonplace hardware worked, or havnig it arbitrarily corrupt memory if the mahematical product is between INT_MAX+1u and UINT_MAX, would favor the latter? How many people would rather jump through hoops in an effort to fit the clang/gcc dialect (which neither compiler actually processes 100% reliably even in strictly-conforming scenarios), versus `-fno-strict-aliasing`, in cases where the latter would allow code to be simpler? – supercat Oct 09 '22 at 06:15
@supercat `-fno-strict-aliasing` is a hallmark of poor code. There is no good reason to ever turn off strict type-based aliasing. – Maxim Egorushkin Oct 13 '22 at 14:32
@MaximEgorushkin: What if one needs to use a buffer to hold different types of objects at different times, because one is using a coding standard that allows allocations only during program startup, and doesn't want to waste RAM using separate buffers for different types? – supercat Oct 13 '22 at 17:29
1

@supercat: That would only be a strict aliasing violation if you do it in a way that actually violates strict aliasing. The standard is 100% fine with creating different objects in the same storage so long as you follow the rules for doing that. I don't agree with Maxim's notion that there's never a reason to use that tool, but I don't accept your notion that the tool is absolutely essential to any low-allocation scenario either. – Nicol Bolas Oct 13 '22 at 17:33
@MaximEgorushkin: Or one wants to, with minimum verbosity, have functions accept pointers to a variety of structure types that share a Common Initial Sequence, and be able to use those pointers to inspect CIS members without having to know or care about which particular types are being passed? Or use machine-specific constructs, when targeting a known architecture, to perform operations based on types' representations more efficiently than would be possible via other means? – supercat Oct 13 '22 at 17:34
@NicolBolas: According to the way clang and gcc interpret the Standard, a region of storage that has been written using two incompatible non-character types may nevermore be read using anything other than character types. If storage is written as type T1 later written as T2, and then read as T2, the read would, per the clang/gcc interpretation, be UB because from the point of view of the T1 write, the read would be a "subsequent access that does not modify the value" and thus the Effective Type during the T2 read would be T1. That may sound absurd, but it's how clang and gcc behave. – supercat Oct 13 '22 at 17:44
@NicolBolas: In both clang and gcc, if code executing a function that write 1 as T1, then one that writes 2 as T2 reads it back as T2, and returns that value, and finally one that writes 3 as T1, then the read-back value (i.e. 2) as T1, and finally reads the storage as T1 and returns the value, the latter function will sometimes erroneously return 1 even though each function only uses the storage as a single type, and writes the storage before reading it. – supercat Oct 13 '22 at 18:36
@supercat: "*the read would, per the clang/gcc interpretation, be UB*" That's not their "interpretation"; that's the literal text of the standard. It's not up to C++ implementations to decide what is and is not UB according to the standard. Those implementations may choose to *define* any particular instance of UB within their implementation, but what is UB as far as the language is concerned is not up to the implementation. If you break the speed limit and get pulled over, but the cop lets you go, you still broke the speed limit even if you weren't punished for it. – Nicol Bolas Oct 13 '22 at 18:43
@supercat: This is why I said "so long as you follow the rules." There are ways to do what you want, but you cannot do them that way and get well-defined behavior according to the standard. All you have to do is create a `T2` in that storage before you do the write. It's only a big deal if you're personally invested in not having to do that. If you're reusing storage, C++ requires you to *say* that you're reusing storage. That's it; that's the rule. – Nicol Bolas Oct 13 '22 at 18:45
@NicolBolas I cannot think of a case in C++ where strict type-based aliasing has to be violated. Using right casts, `char` arrays and `memcpy` in C++ allows even the lowest level code to do type gymnastics without violating strict type-based aliasing. Could you give a contra-example, please? – Maxim Egorushkin Oct 13 '22 at 21:53
For example, the infamous union cast is unnecessary to load a `double` into `int` and back, https://godbolt.org/z/YdMoW5c3q . The example adheres to strict type-based aliasing and the generated assembly is as good as it gets. – Maxim Egorushkin Oct 13 '22 at 22:05
@supercat If you'd like to post an example where you think breaking strict type-based aliasing is necessary for some reason, I would be interested to have a look and make a standard-conforming version of it, without sacrificing run-time performance and/or readability/maintainability. – Maxim Egorushkin Oct 13 '22 at 22:15
@MaximEgorushkin: Suppose one has two functions `useArray1(long long *p)` and `useArray2(long *)` whose code can't be modified because it is under configuration mangagement, and one wishes to repeatedly pass the same array between them, on a platform where both types have the same size and representation. How should one accommodate that reliably and efficiently without -fno-strict-aliasing? – supercat Oct 13 '22 at 22:23
@MaximEgorushkin: See https://godbolt.org/z/M4ze6YME6 for an example. This code will actually work just fine if either `smallish` is smaller than `biggish`, or if `-fno-strict-aliasing is used`, but will otherwise fail on both clang and gcc. – supercat Oct 13 '22 at 22:26
@supercat [Demonstration of aliasing resulting into worse assembly](https://godbolt.org/z/WbeK87W5K). `-fno-strict-aliasing` results in worse assembly everywhere because all pointers are assumed to alias. – Maxim Egorushkin Oct 13 '22 at 22:31
@MaximEgorushkin: A compiler which, given `void test1(T1 *p1, T1 *p2)` was able to recognize that `*(T2)p2` might alias `*p1`, without regard for the specific types `T1` and `T2`, would for many purposes be able to generate more efficient and reliable code than one which, given `void test2(T1 *p1, char *p2)`, would accommodate the possibility that `*p2` might alias `*p1`. The vast majority of situations which seem to involve the "character type exception" actually involve the first pattern, with `T2` being `char`. – supercat Oct 13 '22 at 22:47
Besides, most of the performance advantages of TBAA could be realized using `restrict` if one limits its use to compilers that treat "based upon" as a transitive relationship, such that given `if (flag) *p = 2;`, the value used in the left-hand side of the assignment would be "based upon" the value that `p` holds outside the "if" statement, regardless of how `flag` was computed. – supercat Oct 13 '22 at 22:53
@supercat [Here is your example amended to abide the strict aliasing](https://godbolt.org/z/KvP8r5YrE), requires no `-fno-strict-aliasing` to produce the expected output. `clang` generates exactly the same assembly as your original code with `-fno-strict-aliasing`; `gcc` should as well, but botches it due to its long-standing code generation bugs. – Maxim Egorushkin Oct 13 '22 at 23:20
@supercat _A compiler which, given `void test1(T1 *p1, T1 *p2)` was able to recognize that `*(T2)p2` might alias `*p1`..._ - thanks for sharing your wishful thinking. However, it has no practical value. The best engineering practice is to adhere to the standard - it may cost you a couple of extra lines here and there in low-level code, but makes the code robust, performant and future proof, requiring no obsucre/expert compiler options. – Maxim Egorushkin Oct 13 '22 at 23:29
1

@supercat Peppering code with `restrict` is error-prone and waste of time. The compiler does a better job as long as you follow the language rules rather than fight them in order to avoid having to type two extra lines in low-level code. Only on rare occasions `restrict` is necessary to dealias pointers of the same type to impove performance, but never for careless type punning like in your contrived example. – Maxim Egorushkin Oct 13 '22 at 23:48
Your own example shows that compilers can't do a brilliant job of aliasing analysis without restrict. As for your "fix" to my example, the presumption is that the `process_smallish` code is used in other places, and *you must use it as it is*. Further, gcc eliminated the memmoves because it was able to see that the main program never looked at anything other than element 0 of the array. If one changes your "fix" to the code to address those issues as at https://godbolt.org/z/qaWrroKe6 gcc generates horribly inefficient code that works, while clang generates "efficient" code that doesn't. – supercat Oct 14 '22 at 16:32
Further, I can't think of any way of interpreting the text of the Standard that would require operation to work with memmove but wouldn't require that it work if code had used assignment through a temporary variable (such assignment doesn't affect the code generated by clang or gcc). If clang and gcc interpret the Standard in such a way that the code doesn't work without memmove, the fact that today's versions of gcc processes it meaningfully when using memmove should be viewed as happenstance. – supercat Oct 14 '22 at 16:44

supercat · Answer 2 · 2022-10-14T22:25:10.137

-2

On compilers where both long and long long are 64-bit types without padding bits, an implementation may at its leisure define type int64_t as a synonym for long, a synonym for long long, a synonym for an extended integer type which will be treated as compatible with both, or a synonym for an extended integer type which is incompatible with both.

The C++ Standard allows, but does not require, that implementations treat types which are representation-compatible as alias-compatible. According to the C++ Draft:

Although this document states only requirements on C++ implementations, those requirements are often easier to understand if they are phrased as requirements on programs, parts of programs, or execution of programs.

If a program targets an implementation or configuration that documents that operations involving representation-compatible types will be processed "in a documented manner characteristic of the implementation"--a treatment which is explicitly provided for in the Standard--then a program which relies upon such types being alias-compatible would have defined behavior on that implementation, and could thus be correct.

If an implementation opts, or is configured, not to define the behavior of such actions, instead processing it in gratuitously useless fashion, then the behavior would not be defined on that implementation, and an attempt to use the code on that platform, rather than one that defined the behavior, would be erroneous.

Because C++ Standard explicitly states that it does not impose requirements on C++ programs, correctness of many program may only be judged in reference to particular implementations. Almost all implementations can be configured to define the behavior of code that relies upon representation-compatible types being alias-compatible, and such code would have defined behavior on such implementations or configurations.

edited Oct 14 '22 at 22:25

answered Oct 07 '22 at 22:39

supercat

77,689
9
166
211

@PiotrSiupa: People most likely object to the idea that the Standard isn't intended to say what constructs are correct or incorrect in non-portable programs, since it would undermine the justification for some compilers' nonsensical behaivor. – supercat Oct 08 '22 at 14:43
C and C++ standards set out _requirements_ in order to guarantee program correctness and portability while not introducing unnecessary run-time costs or restricting optimization opportunities. That's why these requirements are as minimalist as possible. Underspecified/unspecified/undefined behaviours are the other side of the minimalism, an engineering trade-off to maximize run-time performance. Other languages do different trade-offs. Checking code constructs for standard compliance is compiler's job, that's why one shouldn't go against the standards with `-fno-strict-aliasing` and such. – Maxim Egorushkin Oct 14 '22 at 01:07
With `-Wall -Wextra` compilers tell one what is wrong with their code. Some developers feel that they know better than compiler writers implementing the standard requirements to the letter, and disable some standard requirements with compiler options to make their sloppy code compile. Such non-compliant sloppy code with its required compiler options become maintenance nightmare because it makes updating/switching compiler or changing the build system a risky/costly affair due to resulting hard to debug regressions. When code requires disabling standard requirements, the code is likely poor. – Maxim Egorushkin Oct 14 '22 at 01:36
@MaximEgorushkin: Have you read the published Rationale for the C Standard at https://www.open-std.org/jtc1/sc22/wg14/www/C99RationaleV5.10.pdf (see page 60). It makes clear that the Standard is intended to allow compilers to perform optimizations *the authors of the Standard recognize would be incorrect* in some cases that wouldn't usually matter, on the presumption that people seeking to write quality compilers would obviously seek to have it process code correctly *in cases that would matter to their customers*, even if the Standard would allow them to do otherwise. – supercat Oct 14 '22 at 16:48
@MaximEgorushkin: It has become fashionable to interpret the phrase "non-portable or erroneous" as meaning "non-portable, and therefore erroneous", ignoring the fact that the authors of the Standard said (page 13) "The goal is to give the programmer a *fighting chance* to make powerful C programs that are also highly portable, without seeming to demean perfectly useful C programs that happen not to be portable...., and said of "undefined behavior" (page 11 "It also identifies areas of possible conforming language extension" What is the source for your claims about the Standard's motivations? – supercat Oct 14 '22 at 16:55
Prior to the ratification of the Standard, C was a collection of dialects, some of which were were more or less suitable than others for accomplishing various tasks on various platforms. If all dialects sharing some trait (e.g. `int` and `long` have the same size and representation) would also share another (e.g. `int` and `long` are alias-compatible) and there was no imaginable reason why any dialect would break that pattern, such implication was very much part of *the language the Committee was chartered to describe*. The Standard makes no effort to enumerate such "popular extensions"... – supercat Oct 14 '22 at 17:12
...and there is I am unaware of any evidence whatsoever that its failure to do so was in any way intended to deny their existence or deprecate the usage thereof. Can you cite any evidence that would suggest such a thing? – supercat Oct 14 '22 at 17:13
I am sorry, I don't find your opinions informative or useful in any way. I am not ever sure what exactly your point. I am interested in using existing compilers for my own benefit with minumum friction and cost, and for that reason I study and align myself with standard-compliant ways of achieving my desired outcomes. Your objections and arguments are irrelevant for me, I prefer expending my energy and attention on creating rather than fighting or bickering about alternative interpretations of underspecified standards verbiage. There are more interesting and rewarding things to do. – Maxim Egorushkin Oct 14 '22 at 20:42

Does reinterpret_cast of an int64_t value really break strict aliasing?

2 Answers2