Custom allocator for std::vector is ignored

Question

I was trying to use a custom allocator for std::vector<char>, but I noticed that std::vector does not need/use any of the member functions from my allocator. How is this possible?

#include <vector>

struct A : private std::allocator<char> {
   typedef std::allocator<char> alloc;
   using alloc::value_type;
   using alloc::pointer;
   using alloc::const_pointer;
   using alloc::difference_type;
   using alloc::size_type;
   using alloc::rebind;
   // member functions have been removed, since the program compiles without them
};

int main() {
    std::vector<char, A> v;
    v.resize(4000);
    for (auto& c : v)
      if (c)
         return 1; // never happens in my environment
   return 0; // all elements initialized to 0. How is this possible?
}

I was trying the above program with an online C++11 compiler (LiveWorkSpace), providing g++ 4.7.2, 4.8 and 4.6.3.

Basically allocate(), deallocate(), construct() and destroy() are not defined in my allocator, yet the program compiles and all the elements will be initialized to 0.

Jonathan Wakely · Accepted Answer · 2013-03-05T13:58:34.453

14

The GCC standard library will always rebind the supplied allocator so internally it does something like this (in C++03):

typedef Alloc::template rebind<value_type>::other _Allocator_type;

(In C++11 it uses allocator_traits but in this case the result is the same.)

The vector then stores an object of that type internally and uses it for all (de)allocation.

Since you haven't defined a rebind member template in your allocator, you've just redeclared the one from the base class, the result of the rebinding is std::allocator<value_type> and not your own type. std::allocator of course provides all those functions, so those are the ones that are used, whether or not you define them on your own type.

You can fix it by adding this to your allocator intead of using alloc::rebind; so that vector stores and uses an A internally:

struct A : private std::allocator<char> {
    template<typename U>
      struct rebind {
        typedef A other;
      };

N.B. this will only work for vector, because vector doesn't strictly need to rebind the allocator (users are required to instantiate the template with allocator<value_type>, but GCC's vector rebinds anyway so that if users instantiate vector<int, std::allocator<char>> it still works.) For node-based containers such as std::set your allocator must be a template that can be rebound, because the container needs to allocate its internal node types, not the value_type, so it needs Alloc::rebind<internal_node_type>::other to be valid.

edited Mar 05 '13 at 13:58

answered Mar 05 '13 at 13:36

Jonathan Wakely

166,810
27
341
521

1

Actually, why is it desirable that it still works if the user provides an `std::allocator` as the allocator for a `vector`? Doesn't it make more sense to get a compiler error? – Andy Prowl Mar 05 '13 at 13:40
1

I'm not sure it is desirable personally, but it's always been that way. For other containers it makes more sense: if a user says `std::map, std::allocator>>` it is a convenience to accept it even though technically the allocator is required to be `std::allocator>` – Jonathan Wakely Mar 05 '13 at 13:42
5

@AndyProwl - allocators really should be template template parameters, but template template parameters didn't exist at the time that the STL (NOTE: **STL**, not **Standard Library**) was designed, so `rebind` was created. – Pete Becker Mar 05 '13 at 13:48
@PeteBecker: So do I understand it correctly, that this is not a template template parameter in C++11 because of backwards compatibility? – Andy Prowl Mar 05 '13 at 13:59
A slight off topic: how will the container internal fields (count, being pointer, etc.) be allocated? Can it be pointed via placement new ? (I assume there is no possibility of specifying new/delete operator for such template) What about allocation of nested containers - will they completely (internal structures + data) be placed in the memory allocated from top most allocator? – Red XIII Mar 05 '13 at 14:25
2

The internal fields are not allocated, they're just members of the object so exist wherever in memory the object exists. Nested containers are no different to any other element type: the object itself and its members will be placed in the memory allocated by the container, if it allocates its own memory (as a nested container will) then that memory will be elsewhere in the address space. See `std::scoped_allocator_adaptor` for a utility that causes elements to inherit the allocator form their container – Jonathan Wakely Mar 05 '13 at 14:30
@JonathanWakely Having `std::vector` how would I put it in an arena? `auto v = new (arena_ptr) std::vector{};`? – Red XIII Mar 05 '13 at 14:37
3

please start your own question instead of hijacking the comments of this one – Jonathan Wakely Mar 05 '13 at 14:43
@AndyProwl - given that `rebind` is part of the allocator, there's not much to gain from changing to a template template parameter. So, yes, backward compatibility. – Pete Becker Mar 05 '13 at 14:48
Wow, I knew about rebind being useful for the node-based containers that don't really allocate `value_type`s but some internal datastructures. But I didn't know `std::vector` is even allowed to rebind and not use the provided allocator, since well, I don't think anyone requires me to rebind my custom allocator to itself. So with `std::vector`'s freedom to rebind *or not* I can in the end never be sure what allocator `std::vector` is really working with if I don't guarantee to rebind to the same allocator. – Christian Rau Mar 05 '13 at 15:07
@ChristianRau, rebinding isn't just "useful" it's essential. If rebinding doesn't work (either via `A::rebind::other` `allocator_traits::rebind_alloc`) your type doesn't meet the allocator requirements, so your program is invalid. If your type meets the req's and rebinding works then the behaviour of GCC's `vector` is invisible and harmless anyway. – Jonathan Wakely Mar 05 '13 at 15:27
@JonathanWakely So do you mean an allocator is **required by standard** to `rebind` to the same allocator type (with a different template argument, of course) and it is completely disallowed to have `A::rebind::other` typedefed to some allocator type `B` totally unrelated to `A`? – Christian Rau Mar 05 '13 at 15:32
1

@ChristianRau, Yes. What's required is that `A1::rebind::other::rebind::other` is the same as `A1` for all `A2`, i.e. you can do a round trip, _including the case where `A1` and `A2` are the same type_. So `A1::rebind::other` must be the same type as `A1`. See Table 28 in [allocator.requirements] – Jonathan Wakely Mar 05 '13 at 15:42

score 7 · Answer 2 · answered Mar 05 '13 at 13:28

7

vector will rebind the allocator. As you bring it into scope from std::allocator, A::rebind<T>::other will simply be std::allocator<T>. So everything works fine.

answered Mar 05 '13 at 13:28

R. Martinho Fernandes

228,013
71
433
510

Custom allocator for std::vector is ignored

2 Answers2

Linked