Custom operator new[] requests for larger than expected memory

Question

I am at a loss as to explain why, in the sample code, the call new Foo [4] to the custom operator new[] requests 68 bytes -- 4 more bytes than it ought to (sizeof(Foo) == 16), whereas the more arcane call Foo::operator new[]( 4 * sizeof( Foo ) ) correctly requests 64 bytes. Note that when the member std::vector<long> m_dummy is removed from Foo both calls correctly request 16 bytes (code on ideone).

#include <vector>
#include <iostream>

struct MemoryManager
{
    static void* allocate( unsigned size )
    {
        static char block[256];
        return block;
    }
};

class Foo
{
public:
    void* operator new[]( size_t size )
    {
        std::cout << "operator new[] : data size -- " << size << std::endl;
        return MemoryManager::allocate( size );
    }  

private:
    std::vector<long> m_dummy;  // Huh?
    unsigned m_num;
};

int main( int argc, char * argv[] )
{
    std::cout << "Foo size: " << sizeof( Foo ) << std::endl;
    new Foo [4];
    Foo::operator new[]( 4 * sizeof( Foo ) );
}

The extra bytes are used to store information about the array you are allocating. That's how `operator delete[]` knows how to perform the cleanup. See Andrei Alexandrescu's CppCon 2015 talk about the issues with such a design, and why it is considered bad: https://www.youtube.com/watch?v=LIb3L4vKZ7U — vsoftco, Oct 14 '15 at 23:07
See http://stackoverflow.com/q/8720425/241631 (possible duplicate even though that question asks about array placement new) — Praetorian, Oct 14 '15 at 23:09
The usual answer to "X does Y unexpectedly" is that your expectations were wrong... And think about destructors. — Kerrek SB, Oct 14 '15 at 23:14

AlexD · Accepted Answer · 2015-10-15T08:11:38.427

5

According to the standard (5.3.4 New):

new T[5] results in a call of operator new[](sizeof(T)*5+x)

Here, x ... are non-negative unspecified values representing array allocation overhead; the result of the new-expression will be offset by this amount from the value returned by operator new[]. ...

The amount of overhead may vary from one invocation of new to another.

In practice it can be used to represent the number of allocated elements. As @vsoftco commented, "That's how operator delete[] knows how to perform the cleanup".

See a footnote in 18.6.1.2 Array forms:

It is not the direct responsibility of operator new[](std::size_t) or operator delete[](void*) to note the repetition count or element size of the array. Those operations are performed elsewhere in the array new and delete expressions. The array new expression, may, however, increase the size argument to operator new[](std::size_t) to obtain space to store supplemental information.

edited Oct 15 '15 at 08:11

answered Oct 14 '15 at 23:05

AlexD

32,156
3
71
65

`y` is kind of missing here... :) – vsoftco Oct 14 '15 at 23:11
@vsoftco Yeah, I was not sure if I should use ellipsis instead, because it makes the sentence less readable :). – AlexD Oct 14 '15 at 23:13
@AlexD : if the extra bytes are possibly used to store the number of allocated elements, why is it that zero extra bytes are sometimes allocated?? And should this decision be taken by the runtime on my behalf when I am completely managing memory? – Olumide Oct 14 '15 at 23:33
1

In which context do you see zero overhead? (In practice I'd expect to see it for every `new T[n]`. Regading your second question, there is a footnote in the standard: _"It is not the direct responsibility of operator `new[](std::size_t)` or operator `delete[](void*)` to note the repetition count or element size of the array. Those operations are performed elsewhere in the array `new` and `delete` expressions. The array new expression, may, however, increase the size argument to operator `new[](std::size_t)` to obtain space to store supplemental information."_ – AlexD Oct 14 '15 at 23:40
@AlexD you should put that last comment as part of the answer, it's really useful – vsoftco Oct 15 '15 at 01:07

Custom operator new[] requests for larger than expected memory

1 Answers1