2

I was implementing a stack allocator and overloaded the operator new[] for a certain class to use my own allocator. I then noticed that operator new[] allocated memory for the amount of elements in the allocated array.
So for example:

test_class* test_arr = new test_class[5];

requests 8 bytes + 5*sizeof(test_class) from my allocator and in the first 8 bytes it stores the size of the array, namely 5 in this case.

Why does it do that? It's the job of my allocator to keep track of the amount of memory that was allocated. For that part it doesn't really make sense, does it? So what's the point? Also, can (or shouldn't I?) I somehow "turn that off"?

Mike van Dyke
  • 2,724
  • 3
  • 16
  • 31
  • The allocator keeps track of memory allocated by storing the size along with the memory allocated. Check out the code for `operator delete[]`. – Mustafa Ozturk Apr 27 '18 at 19:47
  • @MustafaOzturk In libstdc++ `delete` only passes the `void*` to `std::free`. So in my example it would pass a pointer to the first `test_class` element. `free` wouldn't know anything about the type of the pointer, so why good does the amount of elements in the array bring? – Mike van Dyke Apr 27 '18 at 19:54
  • 2
    I'm no expert, but I'd think that if your class has a non-trivial destructor, delete[] needs to know how many elements to call it on – happydave Apr 27 '18 at 19:56
  • `delete[]` not `delete`. To free up an array in C++ you need to use `delete[]`. So, to properly delete the allocation above, you'd have to call `delete[](test_arr)`. – Mustafa Ozturk Apr 27 '18 at 19:56
  • I believe many many C++ call the wrong operator when deleting an array. – Mustafa Ozturk Apr 27 '18 at 20:00
  • @MustafaOzturk `delete[]` just calls `delete`: [see here](https://gcc.gnu.org/viewcvs/gcc/trunk/libstdc%2B%2B-v3/libsupc%2B%2B/del_opv.cc?view=markup) – Mike van Dyke Apr 27 '18 at 20:01
  • 1
    `operator new[]` stores the number of objects in the array so that `operator delete[]` knows how many objects have to be destroyed. – Pete Becker Apr 27 '18 at 20:06
  • @PeteBecker Ok, that makes sense. Do you know if I can be sure that it is always going to be 8 bytes at the beginning of the allocated memory? And also, if I just pass a `void*` ptr, how does `delete[]` know which destructors to call? – Mike van Dyke Apr 27 '18 at 20:14
  • @MikevanDyke, I suppose you should treat that 8 bytes as sizeof(size_t) which is unsigned counter for containers (4 bytes at 32 bit mode, 8 bytes as 64 bit mode etc.) – Yury Schkatula Apr 27 '18 at 21:24
  • Also, you may find this discussion useful too https://stackoverflow.com/questions/18578643/getting-dynamically-allocated-array-size – Yury Schkatula Apr 27 '18 at 21:27
  • Whoops, I said it wrong: it's `new[]` that stores the number of objects, so that `delete[]` knows how many destructors to call. It's entirely up to the implementation how that information is stored; typical implementations store the count in front of the objects, but that's not required, nor is consistency required. – Pete Becker Apr 27 '18 at 21:45
  • @PeteBecker Your two comments answered my questions the best. Could you please combine them into an answer, so I can accept it? – Mike van Dyke Apr 28 '18 at 06:59

2 Answers2

5

When you write p = new T[N] in your code the compiler generates code that calls operator new[] to allocate enough memory for N objects of type T plus whatever bookkeeping information it needs. When you subsequently call delete[] p the compiler calls the destructor for each of the N elements in the array that p points to, then calls operator delete[] to release the memory that it got from operator new[].

But N doesn't always have the same value. You could do p = new T[3]; delete[] p; p = new T[4]; delete[] p;, and the two deletes would each run a different number of destructors.

In order to call the right number of destructors, there has to be a note somewhere recording how many objects are in the array that p points to. That note is usually stored in the bookkeeping part of the memory that the compiler got from the call to operator new[]. That's why it needs the extra space.

That's a typical implementation these days, but the compiler isn't required to do it that way. For example, at least one early implementation kept a separate table of pointer values and destructor counts.

Further, many implementations don't use the extra overhead for types that don't have destructors. So int *p = new int[3] would simply call operator new(3*sizeof(int), and delete[] p would simply call operator delete(p).

Pete Becker
  • 74,985
  • 8
  • 76
  • 165
2

Most likely new T[N] will request a pointer p to memory of size sizeof(size_t) + N * sizeof(T) bytes, store N in the first sizeof(size_t) bytes, then return (T*)(((size_t*)p)+1) to the user (so the first T is located at the returned pointer. )

Then delete[] will take the pointer given, look at *(p-sizeof(size_t))to see how many objects to destroy, destroy them, and pass ((size_t*)p)-1 to the underlying allocator to be freed.

Paul Belanger
  • 2,354
  • 14
  • 23
  • Seems that way, but is that specified by the standard? Also a `void*` is passed to `delete[]`, how does delete know which class destructors to call? – Mike van Dyke Apr 27 '18 at 21:05
  • @MikevanDyke: The destructors have already been called by the time it is passed to `delete[]`. – jxh Apr 27 '18 at 21:06
  • 1
    Be careful with fake pointer arithmetic: `*(p-sizeof(size_t))` doesn't make sense unless `p` has type `char*`. For any other pointer type `p-sizeof(size_t)` points into the weeds, and the behavior is undefined. – Pete Becker Apr 27 '18 at 21:48
  • 1
    You could do `(size_t *)p-1` instead. – jxh Apr 27 '18 at 22:59
  • @jxh -- maybe. My point was more to suggest **talking about** how it works instead of trying to cobble up a code snippet that may or may not show what's actually going on. – Pete Becker Apr 28 '18 at 12:35