0

I am aware that the size of a pointer is fixed (not the size of the data it points to). Now given that, supposing I have a vector of data in global scope and I declare a pointer to a vector of pointers to some/all of that global data, how much memory will this additionally use (RAM)? I am totally at a loss here and would certainly appreciate some ideas/advice. I am hoping that the additional memory consumption would be just the size of vector times the fixed size of a pointer (say, 8 bytes) regardless of the complexity of the data; since the data it points to exists in global scope (i.e., no new data is being allocated; hope I've explained my thoughts clearly).

Some pseudo code that expresses my question would look like: 

std::vector<data> global_data;
std::vector<data*>* my_data = new std::vector<data*>();  //-Is this even valid? 
//-Now populate the vector with pointers to data in global_data
for(int i=0; i<global_data.size(); ++i) {
   my_data->push_back(&global_data[i]);
}

Then my question is how much additional memory in RAM (i.e., in excess of what's already in use from creating global_data) is used up by m_data??

squashed.bugaboo
  • 1,338
  • 2
  • 20
  • 36
  • Don't.... use... `vector`s... of pointers... or pointers... to `vector`s. It is almost always the wrong thing to do. You do realize that it prevents the `vector` from managing memory for you, right? – Ed S. Mar 20 '12 at 23:53
  • Ed.. I am sure that there are ways to work around that concern (at any rate, it is not a concern for my particular purpose).. but go on, please let me know by some examples what exactly you mean. – squashed.bugaboo Mar 21 '12 at 00:05
  • 1
    So, the whole point of a `vector` is that it manages dynamic allocation and deallocation. When its destructor runs it calls `delete` on the items that it has dynamically allocated. However, if you are storing pointers, then it is *allocating pointers*, not the objects it points to, so the will not be deleted when the `vector` goes out of scope. Similarly, if you store a pointer to a `vector` then you have to manually call `delete` on it. This is called RAII (any C++ programmer needs to understand RAII) and you are circumventing the `vector`s ability to safely manage memory for you. – Ed S. Mar 21 '12 at 00:13
  • You're just not using it correctly. The `vector` itself is small and cheap, no need to dynamically allocate it. It dynamically allocates its elements behind the scenes, so don't worry about it. – Ed S. Mar 21 '12 at 00:15
  • @Ed.. that doesn't pose a concern for me. I don't mind managing that memory myself at all. In the snippet, I just showed the part that explains my question.. of course I realize I need to delete all of it, when I am done. No worries (I could even use a smart pointer container from boost if I wanted that taken care of for me). My questions here were focussed on memory usage. – squashed.bugaboo Mar 21 '12 at 00:17
  • 1
    let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/9117/discussion-between-user545591-and-ed-s) – squashed.bugaboo Mar 21 '12 at 00:17
  • @Ed..I disagree with your assessment that maintaining a vector of pointers is "almost always" the wrong thing to do. Maintaining a vector of pointers is especially useful as an ordered list of polymorphic objects. As long as one manages memory correctly, I don't think it's an issue. – Ed J Mar 25 '12 at 12:52

5 Answers5

7

my_data will occupy sizeof(std::vector<data*>*) bytes, and no more than sizeof(void*).

*my_data will occupy sizeof(std::vector<data*>), which can be as little as 3 * sizeof(data**), i.e. no more than 3 * sizeof(void*).

*my_data will manage sizeof(data*) * my_data->capacity() bytes of dynamic memory.

I have grave doubts that that information will be useful in practice.

Kerrek SB
  • 464,522
  • 92
  • 875
  • 1,084
2

If I understand your question right, you want to know how much memory your vector consumes. This will be the fixed size of a pointer (32 or 64 bit) times the amount of pointers stored in it, plus some extra bytes for the size field and stuff. So in most cases you won't recognise the memory it uses.

However, you should definitly not do it like you did in your code example. Because a std::vector my change the position where it saves your data. E.g. if you push_back new objects, it eventually will have to allocate new memory and copy the data to the newly allocated space, since it guarantees that the data space is continues. It then will free the memory used bevore. Your old Pointers will then point into space which is not used by your programm any more causing segmentation faults if you try to use them.

Haatschii
  • 9,021
  • 10
  • 58
  • 95
  • That's a good warning about push_back possibly reallocating. Using reserve() before reading the pointers may address that issue (or maybe the OP's code won't have the issue if it populates the "main" array before the array of pointers). – John Zwinck Mar 20 '12 at 23:48
  • @Haatschii.. I understand this is a bad practise and all. But I've run out of other ideas to optimize the code. I am doing it at a point that's safe enough (there'll be no changes to global_data). Beside the point anyway.. – squashed.bugaboo Mar 20 '12 at 23:50
1

Due to "virtual memory" it is not so simple to say how much "RAM" will be used, but we can talk about how much virtual memory will be consumed. And the answer is roughly as you expected, plus a bit more:

  • Number of elements in the vector * size of each (i.e. size of a pointer).
  • Plus some extra "capacity" in the vector that it uses to avoid constant resizing (probably some small constant factor of the size, and you can query it by calling capacity()).
  • Plus the implementation details of the vector, which might be e.g. three pointers (begin, end, begin + capacity).

If you wanted to express that in C++, you might do something like this:

sizeof(vector<data*>) + my_data.capacity() * sizeof(data*);

Note that this just gives you a rough guess, and ignores more complex pieces like whether more "RAM" is needed to actually do the mapping of the memory you are using in application space, and the behavior of the standard allocator on your system, etc. But as far as C++ and virtual memory are concerned, I think it's a reasonable approximation.

John Zwinck
  • 239,568
  • 38
  • 324
  • 436
  • Appreciate the succinct response. This sounds very correct to me (at least it is what I was hoping to hear). Before I accept it though (due to my own lack of thorough knowledge of C++), I'll wait for other user's to comment a bit. Thanks again. – squashed.bugaboo Mar 20 '12 at 23:45
1

Re: I am hoping that the additional memory consumption would be just the size of vector times the fixed size of a pointer (say, 8 bytes) regardless of the complexity of the data; since the data it points to exists in global scope (i.e., no new data is being allocated; hope I've explained my thoughts clearly).

You are basically right. For any reasonable implementation of std::vector, the underlying storage for a std::vector<T> is just a compact array of T. In this case, your T is the pointer type data * and not data.

There may be some extra storage for efficient expansion, oterhwise every push_back operation would have to grow the array. (Take a look a the reserve and capacity functions of std::vector.)

And of course there is some small overhead for allocating the vector object itself.

Kaz
  • 55,781
  • 9
  • 100
  • 149
1

The memory consumption of the vector is close to what you guessed. You're right that the size of the vector will be independent of the layout of data.

As for actually calculating the total size of the vector, there are two parts that contribute:

  1. The actual vector object itself. A vector has some internal structure, for example, it has a pointer to it's internal array. Also, all its field are aligned to certain values. The total size of any vector (including the alignment) can be found using sizeof(vector<data*>). On my machine this gives 24

  2. The size of the internal array. This is not based on the size() of the vector, but is actually dependent on the capacity() of the vector. The total size will be my_data->capacitiy() * sizeof(data*). I believe there is no padding between array elements, but if there is, this will also have to be accounted for.

The total memory consumption is then just the sum of #1 and #2.

Ken Wayne VanderLinde
  • 18,915
  • 3
  • 47
  • 72