4

I'm writing my own memory system in C++ (for performance reasons, additional debugging information, and so that I can allocate memory that is 16-byte aligned), and I am encountering an issue with new[].

It seems that calling new[] results in an allocation of an additional 4 bytes indicating the number of elements in the array, which throws off the alignment of all subsequent objects. So my question is this:

Is there any way to turn off the usage of these 4 extra bytes with a compiler flag, pragma declaration, etc.?

Here's an example:

// Matrix class has to be 16-byte aligned
Matrix* transforms = new( matrixHeap, ALIGN_16, __FILE__, __LINE__ ) Matrix[31];
transforms[0] = Matrix::Identity;

When looking in the Visual Studio 2013 debugger, I see these values:

returned by new    0x0F468840
transforms         0x0F468844

Finally, I take a peek at the raw memory, and I see this:

0x0F468840   1F 00 00 00
0x0F468844   01 00 00 00

The memory at ******40 is not the first value in transforms[0] as expected. Rather, it contains the value 31. This means that the matrices are only 4-byte aligned. Is there any way to turn off that array size business so that the first byte of transforms[0] ends up in the address returned by new[]?

Here is the important bits of my operator new[]:

void* operator new[] ( size_t size, Heap* heap, uint32_t alignment, char* file, int lineNumber )
{
    size_t alignedSize = size + alignment - 1;
    void* unalignedPtr = heap->Allocate( alignedSize );
    void* alignedPtr = (void*) (((size_t) unalignedPtr + alignment - 1) & ~(alignment - 1));
    return alignedPtr;
}
Vishnu CS
  • 748
  • 1
  • 10
  • 24
David Christo
  • 876
  • 5
  • 12
  • C++ needs to know the size of the allocated array. This is a fundamental aspect of the language. Although C++ does not specify that it must be done in this manner, this is a common approach. Try writing a custom container allocator, rather than a `new` allocator. – Sam Varshavchik Mar 17 '16 at 01:05
  • 1
    I'm not sure I understand the alignment issue you're describing. C++ requires that memory returned by `new` must be aligned by the object's alignment requirements. – templatetypedef Mar 17 '16 at 01:05
  • Well, I wrote custom new and new[] operators that replace the existing operators, and I ensure that my operators return memory with the alignment I need for the structures I create. My issue is that C++ seems to just magically change the returned value. – David Christo Mar 17 '16 at 01:08
  • The redundant bytes might be used as a guard. This also provides "metadata" about your allocated memory. – duong_dajgja Mar 17 '16 at 01:12
  • @VeronikaPrüssels Code added – David Christo Mar 17 '16 at 01:20
  • What is `matrixHeap` ? – M.M Mar 17 '16 at 01:27
  • @M.M The Heap type is something I wrote as I was having some issues with Win32 heaps and memory contiguity. For all intents and purposes, the Heap type works just like a Win32 heap, except that all of the memory allocations in my Heap end up in contiguous virtual memory. – David Christo Mar 17 '16 at 01:32
  • The issue is explained [by this thread](http://stackoverflow.com/questions/8720425/array-placement-new-requires-unspecified-overhead-in-the-buffer) (with `y = 4`). You should also observe that `size` that your `operator new[]` receives is 4 bytes larger than `31*sizeof(Matrix)`. – M.M Mar 17 '16 at 01:39
  • You can't change this (not within Standard C++ anyway), the question would be whether it is possible to work around it somehow. Maybe a more on-point question title would be "How to do custom alignment with operator new[]()" – M.M Mar 17 '16 at 01:40
  • if `sizeof(Matrix)` is a multiple of 16 then you can detect this by inspecting `size % alignment`. If it isn't then you might be out of luck, there is no notion in C++ of an array having stricter alignment requirements than the array elements – M.M Mar 17 '16 at 01:44
  • 1
    Alright, it is ugly, but I guess I have no choice but to replace my new[] calls with a malloc-style version instead. – David Christo Mar 17 '16 at 01:50

1 Answers1

0

Since it seems to consistently use 4 bytes for the size, you could just arrange for your operator new[] function to return 4 less than a 16-byte aligned address. You'd still want to have operator new return a 16-byte aligned address directly.

Chris Dodd
  • 119,907
  • 13
  • 134
  • 226