12

A popular macro-based generic implementation of a vector in C (https://github.com/eteran/c-vector/blob/master/vector.h) uses the following memory layout.

+------+----------+---------+
| size | capacity | data... |
+------+----------+---------+
                  ^
                  | user's pointer

This allows for a very convenient API, where the user gets a vector by simply declaring a pointer of the required type.

float *vf = NULL;
VEC_PUSH_BACK(vf, 3.0);

int *vi = NULL;
size_t sz = VEC_CAPACITY(vi);

Internally, the library accesses the size and capacity like this

#define VEC_CAPACITY(vec) \
    ((vec) ? ((size_t *)(vec))[-1] : (size_t)0)

But isn't that a violation of strict-aliasing?

Shoaib Ahmed
  • 424
  • 2
  • 9
  • Maybe it is popular, but I don't know what these macros are doing, where they are coming from and the Google can't tell me either. – Eugene Sh. Jul 29 '19 at 19:44
  • 1
    The point is these elements are being accessed for both reading and writing as `size_t*`, which is "temporarily" casted to `float*`. I think this is not portable, but well defined withing the implementation where these pointers can be casted to each other without loss of information.. – Eugene Sh. Jul 29 '19 at 19:54
  • So, you mean to say that because the capacity (& size) field is only ever dereferenced as a `size_t`, it is not a violation of strict-aliasing? Right? – Shoaib Ahmed Jul 29 '19 at 20:03
  • 1
    Yes, this is my understanding. But let's wait for an answer with references. – Eugene Sh. Jul 29 '19 at 20:05
  • 2
    It is not a violation of strict aliasing because the `size_t`s do not alias the floats. I'd be more worried about alignments though. – Antti Haapala -- Слава Україні Jul 29 '19 at 20:22
  • 1
    Yes, alignment seems like the real issue. (If the initial assignment of these pointers is also wrapped -- as presumably it must be, to add `2*sizeof(size_t)` to the requested size -- it could be taken care of there.) – Steve Summit Jul 29 '19 at 20:33
  • The only reference you will get is [C11 Standard - §6.5 Expressions (p6,7)](http://port70.net/~nsz/c/c11/n1570.html#6.5p6) -- clear as mud. But here, the cast is just used to establish a type-size so an offset to either `capacity` or `size` can be calculated. A macro implementation is dubious and non-portable. Better as a set of functions with an inline attribute. You see extensive use of macros in code that does NOT have to be portable, like the Linux kernel code -- there is no question that will only be used on Linux. – David C. Rankin Jul 29 '19 at 20:34
  • Thank you all for your insightful comments. I get why this isn't a violation of the rule, but I still don't follow the problem with alignment and how it can be solved. Maybe I need to read more on alignment. Also, I still believe the choice of macros is nice as it makes the interface generic. – Shoaib Ahmed Jul 29 '19 at 20:43
  • You can see the problems very quickly on x86_64 if you do `int *v = NULL; for (int i = 0; i < 10; i++) vector_push_back (v, i); to store `0-9`, then try and erase the odd elements, e.g. `for (int i = 1; i < 10; i+=2) vector_erase (v, i);` Instead of erasing the odd elements, you are left with `0 2 3 5 6 8`. `size_t` and `int` are of different size. – David C. Rankin Jul 29 '19 at 21:21
  • @DavidC.Rankin If that's the problem, that's not an alignment problem, that's a the-macros-are-totally-broken,-not-generic-at-all problem! – Steve Summit Jul 29 '19 at 21:23
  • 1
    The macros also assume `2 * sizeof( size_t )` is a multiple of `_Alignof( max_align_t )`. Are there any 32-bit platforms with 32-bit `size_t` where `_Alignof( max_align_t )` is 16? – Andrew Henle Jul 29 '19 at 21:31
  • You see what it is doing. It is actually shifting the values down as they are erased on each iteration (which is technically fine), but can easily catch you off-guard. Not to mention if you compile with `-pedantic` you are receiving comparison between signed/unsigned warnings, etc.. – David C. Rankin Jul 29 '19 at 21:34
  • To insure proper alignment, code could make a union of `max_algn_t` and a `struct` of the 2 `size_t`. – chux - Reinstate Monica Jul 29 '19 at 23:23

3 Answers3

4

The way this library handles memory does not violate strict aliasing.

Although not mentioned by name in the C standard, strict aliasing basically means you can't access an object of one type as if it were an object of another type. These rules are spelled out in section 6.5, paragraphs 6 and 7:

6 The effective type of an object for an access to its stored value is the declared type of the object, if any. 87) If a value is stored into an object having no declared type through an lvalue having a type that is not a character type, then the type of the lvalue becomes the effective type of the object for that access and for subsequent accesses that do not modify the stored value. If a value is copied into an object having no declared type using memcpy or memmove , or is copied as an array of character type, then the effective type of the modified object for that access and for subsequent accesses that do not modify the value is the effective type of the object from which the value is copied, if it has one. For all other accesses to an object having no declared type, the effective type of the object is simply the type of the lvalue used for the access.

7 An object shall have its stored value accessed only by an lvalue expression that has one of the following types: 88)

  • a type compatible with the effective type of the object,
  • a qualified version of a type compatible with the effective type of the object,
  • a type that is the signed or unsigned type corresponding to the effective type of the object,
  • a type that is the signed or unsigned type corresponding to a qualified version of the effective type of the object,
  • an aggregate or union type that includes one of the aforementioned types among its members (including, recursively, a member of a subaggregate or contained union), or
  • a character type.

87) Allocated objects have no declared type.

88) The intent of this list is to specify those circumstances in which an object may or may not be aliased.

For example, the following violates strict aliasing:

float x = 3.14;
unsigned int *i = (unsigned int *)&x;
printf("value of x: %f, representation of x: %08x\n", x, *i);

Because it tries to read a float as if it were an int.

The way the vector library works does not attempt to do this.

Let's take a look at how a vector is created by the library:

#define vector_grow(vec, count) \
do {                                                                                    \
    if(!(vec)) {                                                                        \
        size_t *__p = malloc((count) * sizeof(*(vec)) + (sizeof(size_t) * 2));          \
        assert(__p);                                                                    \
        (vec) = (void *)(&__p[2]);                                                      \
        vector_set_capacity((vec), (count));                                            \
        vector_set_size((vec), 0);                                                      \
    } else {                                                                            \
        size_t *__p1 = &((size_t *)(vec))[-2];                                          \
        size_t *__p2 = realloc(__p1, ((count) * sizeof(*(vec))+ (sizeof(size_t) * 2))); \
        assert(__p2);                                                                   \
        (vec) = (void *)(&__p2[2]);                                                     \
        vector_set_capacity((vec), (count));                                            \
    }                                                                                   \
} while(0)

And suppose it is called like this:

int *v = NULL;
vector_grow(v, 10);

Because v is NULL, the if part of the macro is entered. It allocates space for 10 int plus 2 size_t. Immediately after the malloc the memory pointed to by __p has no type. Then it assigns to vec:

(vec) = (void *)(&__p[2]);

First, __p is defined as size_t *, so &__p[2] creates a pointer to a location after 2 objects of type size_t, casts that pointer to void *, and assigns it to vec. At this point, none of the allocated memory has a type yet. Next vector_set_capacity is called:

#define vector_set_capacity(vec, size)   \
do {                                     \
    if(vec) {                            \
        ((size_t *)(vec))[-1] = (size);  \
    }                                    \
} while(0)

This first casts vec to a size_t *, which is the original type of __p, and indexes element -1. This is valid because ((size_t *)(vec))[-1] is the same as __p[1]. Now a value of type size_t is written here, so the sizeof(size_t) bytes starting at from __p[1] contains an object of type size_t.

Similarly for vector_set_size:

#define vector_set_size(vec, size)      \
do {                                    \
    if(vec) {                           \
        ((size_t *)(vec))[-2] = (size); \
    }                                   \
} while(0)

((size_t *)(vec))[-2] is the same as __p[0], and writing there also creates an object of type size_t.

So now the memory looks like this:

+--------+----------+---------+
| size_t | size_t   | untyped |
+--------+----------+---------+
^        ^          ^
|        |          |
__p[0]   __p[1]     __p[2]==vec

Now when a user uses vector_push_back it does this:

vec[vector_size(vec)] = (value);

Which works the same as writing to any allocated memory space.

So because __p[0] and __p[1] are only accessed via a size_t *, there is not strict aliasing violation.

One thing that is a problem however is alignment. Memory returned from malloc is suitably aligned to handle data of any type. However, when creating different object in this allocated memory without using a struct those objects might not be properly aligned.

Let's take as an example a system where both int and size_t are 2 bytes in size, and assume a block of memory returned from malloc has an offset of 0. Now we create a vector of type long long, which is at least 8 bytes in size. After creating the vector, the first size_t sits at offset 0 and the second at offset 2. This is fine, because the offset of each is a multiple of the size. However, this means the vector data starts at offset 4. This is not a multiple of 8, so an object of type long long would be misaligned here.

The alignment issue can be resolved by creating a union of max_align_t and a struct of two size_t:

union vector_meta {
    struct {
        size_t size;
        size_t capacity;
    }
    max_align_t align[2];
};

Then vec would be created like this:

union vector_meta *__p = malloc((count) * sizeof(*(vec)) + (sizeof(union vector_meta)));
assert(__p);
(vec) = (void *)(&__p[1]);

And you would access the size and capacity as:

((union vector_meta *)vec)[-1].size
((union vector_meta *)vec)[-1].capacity

This ensures that the memory after the metadata header is aligned properly for any use, and that the size and capacity fields can be accessed safely.

Pablo
  • 13,271
  • 4
  • 39
  • 59
dbush
  • 205,898
  • 23
  • 218
  • 273
2

There isn't an aliasing problem, because the two cells at the beginning of the object are always accessed as size_t.

The library has an alignment problem, however. It assumes that a pointer obtained from malloc which is displaced by 2 * sizeof (size_t) bytes is still suitably aligned for any object type.

This is quite likely true on mainstream architectures, but it's not a standard-defined guarantee. A way to address that would be to define some constant that can be tweaked, like:

#define VEC_HEADER_SIZE (2*sizeof(size_t)) // redefine if insufficient for alignment

The two cell header can then obtained using (size_t *)((char *)(vec)-VEC_HEADER_SIZE), which can then be indexed using [0] and [1] to get at the two size_t cells.

Kaz
  • 55,781
  • 9
  • 100
  • 149
1

The part of the Standard that would cause problems with this kind of code is not the "strict aliasing rule", but rather the specification of pointer arithmetic. The behavior of + and - on pointers is only defined in cases where the original pointer and the result would both point within or "just past" the "same array object", but the Standard is rather vague as to what "array object" is identified by a pointer that is cast from a pointer of another type.

Given e.g.

struct foo { int length; int dat[10]; };
void test(struct foo *p, int index)
{
  if (index < p->length) p->dat[index]++;
  return p->length;
}

the Standard does not require that an implementation allow for the possibility that index might be -1, p->dat-1 might yield the address of p->length, and consequently p->length might get incremented between the if and the return. The way subscripting is defined, however, the code would be equivalent to:

struct foo { int length; int dat[10]; };
void test(struct foo *p, int index)
{
  int *pp = p->dat;
  if (index < p->length) pp[index]++;
  return p->length;
}

which would in turn be equivalent to:

struct foo { int length; int dat[10]; };
void test(struct foo *p, int index)
{
  int *pp = (int*)&p->dat;
  if (index < p->length) pp[index]++;
  return p->length;
}

which starts to look very similar to what you're doing. An implementation which is suitable for low-level memory management should have no trouble processing such code usefully, but the Standard makes no attempt to forbid implementations which are specialized for tasks that don't involve low-level memory management from making assumptions that would render them unsuitable for tasks that do.

supercat
  • 77,689
  • 9
  • 166
  • 211
  • However, this library doesn't declare a formal `struct` type with an array, does it? – Kaz Jul 30 '19 at 19:42
  • @Kaz: There's still a question of what "array object" is identified by a given pointer. If e.g. one is using a type T that is twice the size of an alignment multiple, but one forms a `T*` that starts one alignment-multiple past the start of an allocated region, arithmetic on the resulting pointer would be valid only if it identified part of a `T[]`. Since there could be no preceding element, it would have to identify the first.element. – supercat Jul 30 '19 at 19:47
  • @Kaz: If one has a `T*` which identifies the first element of a `T[]`, how could casting it to `int` yield any defined behavior other than yielding a pointer to the first element of an `int[]`? – supercat Jul 30 '19 at 19:50
  • @Kaz: The purpose of the struct type in this example was to illustrate a scenario where the "same array object" rule would allow useful optimizations that would not be possible if the requirement were merely that a pointer operation be confined to a single allocated chunk. Nothing in the Standard would limit application of that rule to such cases, but present compilers optimize for them. There are various ways of interpreting terms to make some corner cases work, but the Standard lacks the concepts necessary to make all corner cases work sensibly without needlessly blocking optimizations. – supercat Jul 30 '19 at 20:07
  • Maybe relevant, [question about the meaning of 'array'](https://stackoverflow.com/questions/30546449/what-is-the-definition-of-array-in-c) – M.M Jul 30 '19 at 20:55
  • @M.M: The Standard overloads the term "object" to refer to a number of related, but distinct, concepts. It describes traits that are applicable to all array objects, but fails to define the term well enough to unambiguously determine whether any particular region of storage is an array object. – supercat Jul 30 '19 at 21:08