20

I need a safe way to alias between arbitrary POD types, conforming to ISO-C++11 explicitly considering 3.10/10 and 3.11 of n3242 or later. There are a lot of questions about strict aliasing here, most of them regarding C and not C++. I found a "solution" for C which uses unions, probably using this section

union type that includes one of the aforementioned types among its elements or nonstatic data members

From that I built this.

#include <iostream>

template <typename T, typename U>
T& access_as(U* p)
{
    union dummy_union
    {
        U dummy;
        T destination;
    };

    dummy_union* u = (dummy_union*)p;

    return u->destination;
}

struct test
{
    short s;
    int i;
};

int main()
{
    int buf[2];

    static_assert(sizeof(buf) >= sizeof(double), "");
    static_assert(sizeof(buf) >= sizeof(test), "");

    access_as<double>(buf) = 42.1337;
    std::cout << access_as<double>(buf) << '\n';

    access_as<test>(buf).s = 42;
    access_as<test>(buf).i = 1234;

    std::cout << access_as<test>(buf).s << '\n';
    std::cout << access_as<test>(buf).i << '\n';
}

My question is, just to be sure, is this program legal according to the standard?*

It doesn't give any warnings whatsoever and works fine when compiling with MinGW/GCC 4.6.2 using:

g++ -std=c++0x -Wall -Wextra -O3 -fstrict-aliasing -o alias.exe alias.cpp

* Edit: And if not, how could one modify this to be legal?

timrau
  • 22,578
  • 4
  • 51
  • 64
cooky451
  • 3,460
  • 1
  • 21
  • 39
  • I'm 99% sure that this isn't legal in C++ (you're reading an inactive union member). I'm not sure about C11, though, whose rules are more relaxed. – Kerrek SB Apr 01 '12 at 12:48
  • 1
    @KerrekSB he does not read an inactive union member. The access to the union member stays a write-access to an lvalue throughout the access. – Johannes Schaub - litb Apr 01 '12 at 12:55
  • Just get rid of that `dummy_union` and there should be no problem, provided that the alignment of the types are compatible. – Johannes Schaub - litb Apr 01 '12 at 12:59
  • @JohannesSchaub-litb: What makes `u->destination` the active member? – Kerrek SB Apr 01 '12 at 13:04
  • @JohannesSchaub-litb I don't really understand.. if I get rid of the dummy union I'm just aliasing the same lvalue with pointers to different types, which is, as I understand 3.10/10, not legal? – cooky451 Apr 01 '12 at 13:05
  • 1
    @KerrekSB it is not the active member. But it does not *read* anything either. And there isn't even an union object existent at all. But since this is not actually specified at all by C++, we cannot talk about it. It's a black hole. Putting the union away makes this have much less noise. – Johannes Schaub - litb Apr 01 '12 at 13:09
  • @cooky451 3.10/10 does not say such a thing. – Johannes Schaub - litb Apr 01 '12 at 13:11
  • @JohannesSchaub-litb 3.10/10 says through which types one can legally access the stored value of an object. Which point applies here when taking out the union? – cooky451 Apr 01 '12 at 13:14
  • The point "the dynamic type of the object". Reading a `double` object through an lvalue of type `double` is fine. – Johannes Schaub - litb Apr 01 '12 at 13:15
  • @JohannesSchaub-litb: You're right, the code in `main` only *writes*. But it is no different than saying `*reinterpret_cast(buf) = 43.1337;`, is it? It's the same aliasing violation. The point is that `buf` is *not* a pointer to a suitable union object. – Kerrek SB Apr 01 '12 at 13:23
  • 1
    And 3.10/10 must not cover write accesses for anything to make sense in clause 3.8/3.10. For example, this is completely legal *if the alignment is compatible*: `int a; float *b = (float*)&a; *b = 1.0f; std::cout << *b;`. You are reading a `float` object by an lvalue of type `float`. – Johannes Schaub - litb Apr 01 '12 at 13:24
  • 1
    @KerrekSB that's not an aliasing violation. It's no different than `*(double*)malloc(sizeof(double)) = 43.1337;`, just that in this case we have guaranteed alignment conditions. In this case, the lvalue prior to the write did not refer to a `double` object either. – Johannes Schaub - litb Apr 01 '12 at 13:26
  • @JohannesSchaub-litb: But `malloc` is guaranteed to return a pointer that's suitable for any type. By contrast, the OP uses an `int*` obtained as the address of an existing `int` object. – Kerrek SB Apr 01 '12 at 13:27
  • 1
    @KerrekSB i don't know what you mean by "that's suitable for any type". Any writable storage, provided that the size and alignment is OK, is suitable for any object. There is nothing in the Standard that constraints them. – Johannes Schaub - litb Apr 01 '12 at 13:29
  • @JohannesSchaub-litb Ah ok, so when I really introduce a new alias it is legal? At least GCC gives no warning. Very interesting. Is there a way to additionally deal with alignment (or at least assert() it or something)? Would be great to finally have a fully conforming solution. – cooky451 Apr 01 '12 at 13:29
  • 2
    @KerrekSB it's crucial that this works because that's how allocators that recycle storage of old objects work. – Johannes Schaub - litb Apr 01 '12 at 13:30
  • @JohannesSchaub-litb: Allocators don't usually get their memory from the underlying memory of automatic objects, though. System allocation functions return more generous pointers than `&x` for some random `x`. – Kerrek SB Apr 01 '12 at 13:32
  • @KerrekSB The current Standard allows code to obtain memory of arbitrary sources and use them to create new objects. But now that you say it, IIRC, there's an issue resolution that renders this undefined (taking the memory of an automatic/static object of non-char type), though it is not part of C++11. – Johannes Schaub - litb Apr 01 '12 at 13:36
  • 1
    @JohannesSchaub-litb: But at the very least the memory that you obtain must be aligned for the type that you want to use. Granted that `int` doesn't have a destructor (with effects), so you *can* use that memory, but you can first and foremost only use it to store other ints in it. Everything else needs further guarantees. – Kerrek SB Apr 01 '12 at 13:38
  • @KerrekSB http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_active.html#1116 . See the "If a program obtains storage for a particular type ..." rule. – Johannes Schaub - litb Apr 01 '12 at 13:46
  • 1
    Note that gcc/g++ makes some additional guarantees about aliasing in an union which are not part of the C++ standard. That is, some union aliasing which is clearly not allowed in the standard is guaranteed to work in gcc/g++. I'm not sure whether this applies here, though. – celtschk Apr 01 '12 at 16:39
  • @celtschk Well, that could explain the gone warning, and be a hint that this way is not (yet) conforming. – cooky451 Apr 01 '12 at 16:42
  • Please take extended discussions to [chat]. – Tim Post Apr 01 '12 at 16:48
  • @cooky451: A conforming compiler is free to define undefined behaviour. Also, for undefined behaviour a diagnostic is not required (and sometimes not even possible). "Undefined behavior" just means that the implementation may do whatever it wants. – celtschk Apr 01 '12 at 17:09
  • @TimPost "_Please take extended discussions to Stack Overflow Chat._" But I think the **conclusion** of the discussion belongs here, doesn't it? – curiousguy Jul 20 '12 at 19:19

4 Answers4

15

This will never be legal, no matter what kind of contortions you perform with weird casts and unions and whatnot.

The fundamental fact is this: two objects of different type may never alias in memory, with a few special exceptions (see further down).

Example

Consider the following code:

void sum(double& out, float* in, int count) {
    for(int i = 0; i < count; ++i) {
        out += *in++;
    }
}

Let's break that out into local register variables to model actual execution more closely:

void sum(double& out, float* in, int count) {
    for(int i = 0; i < count; ++i) {
        register double out_val = out; // (1)
        register double in_val = *in; // (2)
        register double tmp = out_val + in_val;
        out = tmp; // (3)
        in++;
    }
}

Suppose that (1), (2) and (3) represent a memory read, read and write, respectively, which can be very expensive operations in such a tight inner loop. A reasonable optimization for this loop would be the following:

void sum(double& out, float* in, int count) {
    register double tmp = out; // (1)
    for(int i = 0; i < count; ++i) {
        register double in_val = *in; // (2)
        tmp = tmp + in_val;
        in++;
    }
    out = tmp; // (3)
}

This optimization reduces the number of memory reads needed by half and the number of memory writes to 1. This can have a huge impact on the performance of the code and is a very important optimization for all optimizing C and C++ compilers.

Now, suppose that we don't have strict aliasing. Suppose that a write to an object of any type can affect any other object. Suppose that writing to a double can affect the value of a float somewhere. This makes the above optimization suspect, because it's possible the programmer has in fact intended for out and in to alias so that the sum function's result is more complicated and is affected by the process. Sounds stupid? Even so, the compiler cannot distinguish between "stupid" and "smart" code. The compiler can only distinguish between well-formed and ill-formed code. If we allow free aliasing, then the compiler must be conservative in its optimizations and must perform the extra store (3) in each iteration of the loop.

Hopefully you can see now why no such union or cast trick can possibly be legal. You cannot circumvent fundamental concepts like this by sleight of hand.

Exceptions to strict aliasing

The C and C++ standards make special provision for aliasing any type with char, and with any "related type" which among others includes derived and base types, and members, because being able to use the address of a class member independently is so important. You can find an exhaustive list of these provisions in this answer.

Furthermore, GCC makes special provision for reading from a different member of a union than what was last written to. Note that this kind of conversion-through-union does not in fact allow you to violate aliasing. Only one member of a union is allowed to be active at any one time, so for example, even with GCC the following would be undefined behavior:

union {
    double d;
    float f[2];
};
f[0] = 3.0f;
f[1] = 5.0f;
sum(d, f, 2); // UB: attempt to treat two members of
              // a union as simultaneously active

Workarounds

The only standard way to reinterpret the bits of one object as the bits of an object of some other type is to use an equivalent of memcpy. This makes use of the special provision for aliasing with char objects, in effect allowing you to read and modify the underlying object representation at the byte level. For example, the following is legal, and does not violate strict aliasing rules:

int a[2];
double d;
static_assert(sizeof(a) == sizeof(d));
memcpy(a, &d, sizeof(d));

This is semantically equivalent to the following code:

int a[2];
double d;
static_assert(sizeof(a) == sizeof(d));
for(size_t i = 0; i < sizeof(a); ++i)
   ((char*)a)[i] = ((char*)&d)[i];

GCC makes a provision for reading from an inactive union member, implicitly making it active. From the GCC documentation:

The practice of reading from a different union member than the one most recently written to (called “type-punning”) is common. Even with -fstrict-aliasing, type-punning is allowed, provided the memory is accessed through the union type. So, the code above will work as expected. See Structures unions enumerations and bit-fields implementation. However, this code might not:

int f() {
    union a_union t;
    int* ip;
    t.d = 3.0;
    ip = &t.i;
    return *ip;
}

Similarly, access by taking the address, casting the resulting pointer and dereferencing the result has undefined behavior, even if the cast uses a union type, e.g.:

int f() {
    double d = 3.0;
    return ((union a_union *) &d)->i;
} 

Placement new

(Note: I'm going by memory here as I don't have access to the standard right now). Once you placement-new an object into a storage buffer, the lifetime of the underlying storage objects ends implicitly. This is similar to what happens when you write to a member of a union:

union {
    int i;
    float f;
} u;

// No member of u is active. Neither i nor f refer to an lvalue of any type.
u.i = 5;
// The member u.i is now active, and there exists an lvalue (object)
// of type int with the value 5. No float object exists.
u.f = 5.0f;
// The member u.i is no longer active,
// as its lifetime has ended with the assignment.
// The member u.f is now active, and there exists an lvalue (object)
// of type float with the value 5.0f. No int object exists.

Now, let's look at something similar with placement-new:

#define MAX_(x, y) ((x) > (y) ? (x) : (y))
// new returns suitably aligned memory
char* buffer = new char[MAX_(sizeof(int), sizeof(float))];
// Currently, only char objects exist in the buffer.
new (buffer) int(5);
// An object of type int has been constructed in the memory pointed to by buffer,
// implicitly ending the lifetime of the underlying storage objects.
new (buffer) float(5.0f);
// An object of type int has been constructed in the memory pointed to by buffer,
// implicitly ending the lifetime of the int object that previously occupied the same memory.

This kind of implicit end-of-lifetime can only occur for types with trivial constructors and destructors, for obvious reasons.

Community
  • 1
  • 1
anttirt
  • 151
  • 3
  • If "two objects of different type may never alias in memory, with a few special exceptions" is a "fundamental fact", how does placement new work? (It's not listed in the exception list.) – cooky451 Apr 02 '12 at 10:55
  • @cooky451 I've added a section on placement new. – anttirt Apr 02 '12 at 13:29
  • This is the best resource I’ve seen on the subject of aliasing and aliasing exceptions. Great answer. – gmbeard May 23 '18 at 19:46
6

Aside from the error when sizeof(T) > sizeof(U), the problem there could be, that the union has an appropriate and possibly higher alignment than U, because of T. If you don't instantiate this union, so that its memory block is aligned (and large enough!) and then fetch the member with destination type T, it will break silently in the worst case.

For example, an alignment error occurs, if you do the C-style cast of U*, where U requires 4 bytes alignment, to dummy_union*, where dummy_union requires alignment to 8 bytes, because alignof(T) == 8. After that, you possibly read the union member with type T aligned at 4 instead of 8 bytes.


Alias cast (alignment & size safe reinterpret_cast for PODs only):

This proposal does explicitly violate strict aliasing, but with static assertions:

///@brief Compile time checked reinterpret_cast where destAlign <= srcAlign && destSize <= srcSize
template<typename _TargetPtrType, typename _ArgType>
inline _TargetPtrType alias_cast(_ArgType* const ptr)
{
    //assert argument alignment at runtime in debug builds
    assert(uintptr_t(ptr) % alignof(_ArgType) == 0);

    typedef typename std::tr1::remove_pointer<_TargetPtrType>::type target_type;
    static_assert(std::tr1::is_pointer<_TargetPtrType>::value && std::tr1::is_pod<target_type>::value, "Target type must be a pointer to POD");
    static_assert(std::tr1::is_pod<_ArgType>::value, "Argument must point to POD");
    static_assert(std::tr1::is_const<_ArgType>::value ? std::tr1::is_const<target_type>::value : true, "const argument must be cast to const target type");
    static_assert(alignof(_ArgType) % alignof(target_type) == 0, "Target alignment must be <= source alignment");
    static_assert(sizeof(_ArgType) >= sizeof(target_type), "Target size must be <= source size");

    //reinterpret cast doesn't remove a const qualifier either
    return reinterpret_cast<_TargetPtrType>(ptr);
}

Usage with pointer type argument ( like standard cast operators such as reinterpret_cast ):

int* x = alias_cast<int*>(any_ptr);

Another approach (circumvents alignment and aliasing issues using a temporary union):

template<typename ReturnType, typename ArgType>
inline ReturnType alias_value(const ArgType& x)
{
    //test argument alignment at runtime in debug builds
    assert(uintptr_t(&x) % alignof(ArgType) == 0);

    static_assert(!std::tr1::is_pointer<ReturnType>::value ? !std::tr1::is_const<ReturnType>::value : true, "Target type can't be a const value type");
    static_assert(std::tr1::is_pod<ReturnType>::value, "Target type must be POD");
    static_assert(std::tr1::is_pod<ArgType>::value, "Argument must be of POD type");

    //assure, that we don't read garbage
    static_assert(sizeof(ReturnType) <= sizeof(ArgType),"Target size must be <= argument size");

    union dummy_union
    {
        ArgType x;
        ReturnType r;
    };

    dummy_union dummy;
    dummy.x = x;

    return dummy.r;
}

Usage:

struct characters
{
    char c[5];
};

//.....

characters chars;

chars.c[0] = 'a';
chars.c[1] = 'b';
chars.c[2] = 'c';
chars.c[3] = 'd';
chars.c[4] = '\0';

int r = alias_value<int>(chars);

The disadvantage of this is, that the union may require more memory than actually needed for the ReturnType


Wrapped memcpy (circumvents alignment and aliasing issues using memcpy):

template<typename ReturnType, typename ArgType>
inline ReturnType alias_value(const ArgType& x)
{
    //assert argument alignment at runtime in debug builds
    assert(uintptr_t(&x) % alignof(ArgType) == 0);

    static_assert(!std::tr1::is_pointer<ReturnType>::value ? !std::tr1::is_const<ReturnType>::value : true, "Target type can't be a const value type");
    static_assert(std::tr1::is_pod<ReturnType>::value, "Target type must be POD");
    static_assert(std::tr1::is_pod<ArgType>::value, "Argument must be of POD type");

    //assure, that we don't read garbage
    static_assert(sizeof(ReturnType) <= sizeof(ArgType),"Target size must be <= argument size");

    ReturnType r;
    memcpy(&r,&x,sizeof(ReturnType));

    return r;
}

For dynamic sized arrays of any POD type:

template<typename ReturnType, typename ElementType>
ReturnType alias_value(const ElementType* const array,const size_t size)
{
    //assert argument alignment at runtime in debug builds
    assert(uintptr_t(array) % alignof(ElementType) == 0);

    static const size_t min_element_count = (sizeof(ReturnType) / sizeof(ElementType)) + (sizeof(ReturnType) % sizeof(ElementType) != 0 ? 1 : 0);

    static_assert(!std::tr1::is_pointer<ReturnType>::value ? !std::tr1::is_const<ReturnType>::value : true, "Target type can't be a const value type");
    static_assert(std::tr1::is_pod<ReturnType>::value, "Target type must be POD");
    static_assert(std::tr1::is_pod<ElementType>::value, "Array elements must be of POD type");

    //check for minimum element count in array
    if(size < min_element_count)
        throw std::invalid_argument("insufficient array size");

    ReturnType r;
    memcpy(&r,array,sizeof(ReturnType));
    return r;
}

More efficient approaches may do explicit unaligned reads with intrinsics, like the ones from SSE, to extract primitives.


Examples:

struct sample_struct
{
    char c[4];
    int _aligner;
};

int test(void)
{
    const sample_struct constPOD    = {};
    sample_struct pod               = {};
    const char* str                 = "abcd";

    const int* constIntPtr  = alias_cast<const int*>(&constPOD);
    void* voidPtr           = alias_value<void*>(pod);
    int intValue            = alias_value<int>(str,strlen(str));

    return 0;
}

EDITS:

  • Assertions to assure conversion of PODs only, may be improved.
  • Removed superfluous template helpers, now using tr1 traits only
  • Static assertions for clarification and prohibition of const value (non-pointer) return type
  • Runtime assertions for debug builds
  • Added const qualifiers to some function arguments
  • Another type punning function using memcpy
  • Refactoring
  • Small example
Sam
  • 7,778
  • 1
  • 23
  • 49
  • 1
    With the second static_assert this wouldn't even allow to cast char* to int* on most machines.. – cooky451 Apr 01 '12 at 16:51
  • @cooky451: The proposed function is exclusively intended to cast unions. – jweyrich Apr 01 '12 at 16:55
  • Actually, I partly misunderstood cooky451, char has most probably less alignment than int, but anyway, alignment safe casts should work now. – Sam Apr 01 '12 at 17:18
  • It still has the same problem. (Or I'm overseeing something here.) -- Consider SourceType to be char and dest_type to be int and the second static_assert will fail. – cooky451 Apr 01 '12 at 17:18
  • This is intended to avoid unaligned reads, since an int may be aligned to four bytes, char to one. If you want to cast 4 chars to an int, you must instantiate your union to get the chars to the same alignment as an int and thats it. Just never return a pointer or a reference to an union member on stack. – Sam Apr 01 '12 at 17:22
  • Well, the cast from char* to int* has to work. If it doesn't compile on weird machines that's ok, but at least the behavior is well defined. (As I hope.) -- But let me ask: This considers alignment, but do you have in mind strict aliasing? – cooky451 Apr 01 '12 at 17:30
  • 1
    Your last edit actually copies the values, which is not what I want. memcpy already does that pretty good. ;) – cooky451 Apr 01 '12 at 17:57
  • 2
    Your answer could be memcpy, but thats no fun :-) And strict aliasing is not an issue here. You absolutely have to copy the source memory somehow, if the alignment possibly isn't right. Runtime alignment checking will lead to nothing. – Sam Apr 01 '12 at 18:01
  • Some updates; Any suggestions for improvement would be appreciated – Sam Apr 03 '12 at 22:19
4

I think that at the most fundamental level, this is impossible and violates strict aliasing. The only thing you've achieved is tricking the compiler into not noticing.

Puppy
  • 144,682
  • 38
  • 256
  • 465
  • So do you think there is just no way to get a "safe way to alias between arbitrary POD types"? This feels kind of awkward to me.. the probably most liberal language I know doesn't allow me to do the things I need to do? :( -- Did you read the comments? What do you say to the suggestions Johannes made? – cooky451 Apr 01 '12 at 16:26
  • 1
    @cooky451: "This feels kind of awkward to me.." That's because what you're trying to do is awkward to C++. – Nicol Bolas Apr 01 '12 at 16:42
  • @NicolBolas Actually, I don't even think that's so unusual. I just want to access the plain memory (which is perfectly valid, shown by the char* extra rule, and shown by the fact that most compilers don't even bother with strict aliasing), while not being limited to char*s. – cooky451 Apr 01 '12 at 16:46
  • 1
    @cooky451: Which is awkward because the rules of C++'s type system don't let you do that. – Nicol Bolas Apr 01 '12 at 16:58
  • @NicolBolas Cool so.. it is basically impossible to implement any performant hashing algorithms in a standard conforming way? Because you need to do integer math there; but the language won't let you do that, because you can only rely on char* to work? There has to be some way. – cooky451 Apr 01 '12 at 17:04
  • 1
    @cooky451: "There has to be some way." No there doesn't; C++ doesn't exact to do everything *exactly* as you desire it to be done. Besides, who says that you can't implement hashing efficiently with `char*`? Just pull sequences of four `char*`s; it's not like you really care about the endian-ness of the object. Either that, or you can just accept that your code lives in implementation-defined land. It's never stopped anyone before... – Nicol Bolas Apr 01 '12 at 17:30
  • @NicolBolas: "It's never stopped anyone before": I like you. This perfectly captures the way people simply keep doing it :) – xtofl Apr 01 '12 at 17:34
  • @NicolBolas Well, you sure want to use shift, rotate and all that good stuff, but I imagine that is painfully slow to emulate with 4 chars. As for living in implementation-defined land: I'm ok with that. But the least I want to reach is an compile time error on systems which have other requirements. – cooky451 Apr 01 '12 at 17:53
  • 2
    @cooky451: Of course there is no safe way to alias between arbitrary types. That's practically the *exact text* of the strict aliasing rule- you can't alias except `char*`. – Puppy Apr 01 '12 at 18:12
  • @cooky451: Re-interpreting a type as characters or integers is *always* going to give platform-dependent results. What you are trying to do can be done efficiently in C++, just not *portably*. (Unless, for example, each type implements its own portable hashing function.) – David Schwartz Apr 01 '12 at 19:19
  • Even though "aliasing" as such is not really possible, the intended effect - reinterpreting a value from one POD type to another - is achievable safely and conformantly using memcpy. See: http://dbp-consulting.com/tutorials/StrictAliasing.html. – Lubo Antonov Apr 01 '12 at 21:42
  • @lucas1024: That does not permit reinterpreting a pointer.. only the value, which is not the intended effect. – Puppy Apr 06 '12 at 12:51
2

My question is, just to be sure, is this program legal according to the standard?

No. The alignment may be unnatural using the alias you have provided. The union you wrote just moves the point of the alias. It may appear to work, but that program may fail when CPU options, ABI, or compiler settings change.

And if not, how could one modify this to be legal?

Create natural temporary variables and treat your storage as a memory blob (moving in and out of the blob to/from temporaries), or use a union which represents all your types (remember, one active element at a time here).

justin
  • 104,054
  • 14
  • 179
  • 226
  • To clarify that. As JohannesSchaub suggested, is this legal aliasing and does not conflict with any rules? http://ideone.com/GBhqV – cooky451 Apr 01 '12 at 17:12
  • @cooky451 it is not legal. here's the write to memory blob approach (of course, you would also need to read from it at some point): `char buf[sizeof(LargestPODType)]; \n int temp(5); /* << the natural temporary */ \n /* now move bytes from temp to buf by casting &temp as a char* - or memcpy */ \n` – justin Apr 01 '12 at 17:31
  • 1
    I agree and my preference is for memcpy, as it leaves no room for ambiguity. Here is a source that discussed the whole issue in some depth: http://dbp-consulting.com/tutorials/StrictAliasing.html. – Lubo Antonov Apr 01 '12 at 21:29