Why doesn't strict aliasing rule apply to int* and unsigned*?

Question

In the C language, we cannot access an object using an lvalue expression that has an incompatible type with the effective type of that object as this yields to undefined behaviour. And based on this fact, the strict aliasing rule states that two pointers cannot alias each other (refer to the same object in memory) if they have incompatible types. But in p6.2.4 of C11 standard, it is allowed to access an unsigned effective type with a signed version lvalue and vice-versa.

Because of the last paragraph two pointers int *a and unsigned *b may alias each other and the change of the value of the object pointed by one of them might lead to the change of the value of the object pointed by the other (Because it is the same object).

Let's demonstrate this on the compiler level:

int f (int *a, unsigned *b)
{
    *a = 1;
    *b = 2;

    return *a;
}

The generated assembly of the above function looks like this on GCC 6.3.0 with -O2:

0000000000000000 <f>:
   0:   movl   $0x1,(%rdi)
   6:   movl   $0x2,(%rsi)
   c:   mov    (%rdi),%eax
   e:   retq  

Which is quite expected because GCC doesn't optimize the return value and still reads the value *a again after the write to *b (Because the change of *b might lead to the change of *a).

But with this other function :

int ga;
unsigned gb;

int *g (int **a, unsigned **b)
{
    *a = &ga;
    *b = &gb;

    return *a;
}

The generated assembly is quite surprising (GCC -O2):

0000000000000010 <g>:
  10:   lea    0x0(%rip),%rax        # 17 <g+0x7>
  17:   lea    0x0(%rip),%rdx        # 1e <g+0xe>
  1e:   mov    %rax,(%rdi)
  21:   mov    %rdx,(%rsi)
  24:   retq 

The return value is optimized and it is not read again after the write to *b. I know that int *a and unsigned *b are not compatible types but what about the rule in paragraph P6.2.4 (It is allowed to access an unsigned effective type with a signed version lvalue and vice-versa)? Why doesn't it apply in this situation? And why does the compiler make that kind of optimization in this case?

There is somthing I don't understand about this whole story of compatible types and strict aliasing. Can someone enlighten us? (And please explain why do two pointers have incompatible types but can alias each other, think of int *a and unsigned *b).

Answer 1

Given int **a and unsigned **b, the type of *a is not the signed or unsigned type corresponding to the effective type of *b, nor is *b the signed or unsigned type corresponding to the effective type of *a. Therefore, this rule permitting aliasing via corresponding signed or unsigned types does not apply. As no other rules permitting aliasing apply either, the compiler is entitled to assume the write to *b does not modify *a, and therefore the value the compiler wrote to *a in *a = &ga; is still present in *a for the return *a; statement.

The fact that int * points to a signed int does not make it a signed type. It is a pointer. int * and unsigned * are pointers to different types. Even if they were considered signed or unsigned, they would be signed or unsigned pointers to different types: If int * were a signed pointer, it would be a signed pointer to int, and the corresponding unsigned version would be an unsigned pointer to int, not any pointer to unsigned.

Answer 2

To understand the intended meaning of the signed/unsigned exemption, one must first understand the background of those types. The C language didn't originally have an "unsigned" integer type, but was instead designed for use on two's-complement machines with quiet wraparound on overflow. While there were a few operations, most notably the relational operators, divide, remainder, and right-shift, where signed and unsigned behaviors would differ, performing most operations on signed types would yield the same bit patterns as performing those same operations on unsigned types, thus minimizing the need for the latter.

Although unsigned types are certainly useful even on quiet-wraparound two's-complement machines, they are indispensable on platforms that do not support quiet-wraparound two's-complement semantics. Because C did not initially support such platforms, however, a lot of code which logically "should" have used used unsigned types, and would have used them if they'd existed sooner, was written to use signed types instead. The authors of the Standard did not want the type-access rules to create any difficulty interfacing between code which used signed types because unsigned types weren't available when it was written, and code which used unsigned types because they were available and their use would make sense.

The historical reasons for treating int and unsigned interchangeably would apply equally to allowing objects of type int* to be accessed using lvalues of type unsigned* and vice versa, int** to be accessed using unsigned**, etc. While the Standard doesn't explicitly specify that any such usages should be allowed, it also neglects to mention some other uses that should obviously be allowed, and thus cannot be reasonably viewed as fully and completely describing everything that implementations should support.

The Standard fails to distinguish between two kinds of circumstances involving pointer-based type punning--those which involve aliasing, and those which don't-- beyond a non-normative footnote saying that the purpose of the rules is to indicate when things may alias. The distinction is illustrated below:

int *x;
unsigned thing;
int *usesAliasingUnlessXandPDisjoint(unsigned **p)
{
  if (x)
    *p = &thing;
  return x;
}

if x and *p identify the same storage there would be aliasing between *p and x, because the creation of p and the write via *p would be separated by a conflicting access to the storage using the lvalue x. However, given something like:

unsigned thing;
unsigned writeUnsignedPtr(unsigned **p)
{ *p = &thing; }

int *x;
int *doesNotUseAliasing(void)
{
  if (x)
    writeUnsignedPtr((unsigned**)&x);
  return x;
}

there would be no aliasing between the *p argument and x, since within the lifetime of the passed pointer p, neither x nor any other other pointer or lvalue not derived from p, is used to access the same storage as *p. I think it's clear the authors of the Standard wanted to allow for the latter pattern. I think it's less clear whether they wanted to allow the former even for lvalues of type signed and unsigned [as opposed to signed* or unsigned*], or didn't realize that limiting application of the rule to cases that actually involve aliasing would be sufficient to allow the latter.

The way gcc and clang interpret the aliasing rules does not extend the compatibility between int and unsigned to int* and unsigned*--a limitation which is allowable given the wording of the Standard, but which--at least in cases not involving aliasing, I would regard as contrary to the Standard's stated purpose.

Your particular example does involve aliasing in cases where *a and *b overlap, since either a was created first and a conflicting access via *b happens between such creation and the last use of *a, or b was created first and a conflicting access via *a happens between such creation and the last use of b. I'm not sure whether the authors of the Standard intended to allow such usage or not, but the same reasons that would justify allowing int and unsigned would apply equally to int* and unsigned*. On the other hand, gcc and clang's behavior does not seem to be dictated by what the authors of the Standard meant to say as indicated by the published Rationale, but rather by what they fail to demand that compilers do.