memory usage by a process in c++ program

Question

In My C++ program I use

* m_Map= new map<int, list<object> >();
delete(m_Map);
m_Map->erase(TxId);

I added 1000000 elements to map and I checked time to time in the loop the memory usage of the process

for(int x=1;x<=1000000;x++){
    m_Map->emplace(txId, OBject);
    if(x%100000==0) {
        process_mem_usage(vm, rss);
        cout << "after add a key  VM: " << vm << "; RSS: " << rss << endl;
        }
    }

then I again print the process RSS memory usage by erasing one by one element from the map

 for(int x=1;x<=1000000;x++){
      m_Map->erase(x);
      if(x%100000==0) {
           process_mem_usage(vm, rss);
           cout << "after earse a key VM: " << vm << "; RSS: " << rss << endl;
        }
 }

using this memory usage function

void process_mem_usage(double& vm_usage, double& resident_set)
{
   using std::ios_base;
   using std::ifstream;
   using std::string;

   vm_usage     = 0.0;
   resident_set = 0.0;

   // 'file' stat seems to give the most reliable results
   //
   ifstream stat_stream("/proc/self/stat",ios_base::in);

   // dummy vars for leading entries in stat that we don't care about
   //
   string pid, comm, state, ppid, pgrp, session, tty_nr;
   string tpgid, flags, minflt, cminflt, majflt, cmajflt;
   string utime, stime, cutime, cstime, priority, nice;
   string O, itrealvalue, starttime;

   // the two fields we want
   //
   unsigned long vsize;
   long rss;

   stat_stream >> pid >> comm >> state >> ppid >> pgrp >> session >> tty_nr
               >> tpgid >> flags >> minflt >> cminflt >> majflt >> cmajflt
               >> utime >> stime >> cutime >> cstime >> priority >> nice
               >> O >> itrealvalue >> starttime >> vsize >> rss; // don't care about the rest

   stat_stream.close();

   long page_size_kb = sysconf(_SC_PAGE_SIZE) / 1024; // in case x86-64 is configured to use 2MB pages
   vm_usage     = vsize / 1024.0;
   resident_set = rss * page_size_kb;
}

I came up with this result I can't understand really.

 Initially VM: 12660; RSS: 1120
after add a key  VM: 28240; RSS: 16960
after add a key  VM: 43816; RSS: 32536
after add a key  VM: 59524; RSS: 48112
after add a key  VM: 75100; RSS: 63688
after add a key  VM: 90676; RSS: 79264
after add a key  VM: 106384; RSS: 95104
after add a key  VM: 121960; RSS: 110680
after add a key  VM: 137672; RSS: 126256
after add a key  VM: 153248; RSS: 141832
after add a key  VM: 168824; RSS: 157408
after earse a key VM: 168824; RSS: 157408
after earse a key VM: 168824; RSS: 157408
after earse a key VM: 168824; RSS: 157408
after earse a key VM: 168824; RSS: 157408
after earse a key VM: 168824; RSS: 157408
after earse a key VM: 168824; RSS: 157408
after earse a key VM: 168824; RSS: 157408
after earse a key VM: 168824; RSS: 157408
after earse a key VM: 168824; RSS: 157408
after earse a key VM: 168824; RSS: 157408


after destroying the map VM: 12672; RSS: 1536

What I feel is it should free the memory when I delete the key,value pairs from the map.but as you can see it will not free the memory untill I delete(free) the map finally

delete(m_Map);

some one can explain how it happens in c++,I looked at c++ map::emplace,erase function documentation.which does not give any clue about this..

Possible duplicate of [Memory not released when emptying std::vector inside a user-defined class](http://stackoverflow.com/questions/38245712/memory-not-released-when-emptying-stdvector-inside-a-user-defined-class) — , Nov 29 '16 at 10:17
Possible duplicate of [Linux Allocator Does Not Release Small Chunks of Memory](http://stackoverflow.com/questions/10943907/linux-allocator-does-not-release-small-chunks-of-memory) — quimnuss, Nov 29 '16 at 10:20

score 0 · Accepted Answer · 2016-11-29T10:34:41.537

0

You can think that processes have two different memory allocators. First one allocator is used when you request memory from the kernel(sbrk(2), mmap(2)). Second one is usespace allocator which slices memory you got from first allocator to your requests from the C++ code.

When you do new/malloc - you request second allocator for memory(which can use first allocator to request more memory from kernel if it cannot fulfill your request). When you do delete/free - you give up memory to second allocator which can(but not must!) return memory to first allocator and to kernel.

So it doesn't necessary mean, that when you do new,malloc/delete,free it will immediately result in changes of RSS or VM.

edited Nov 29 '16 at 10:34

answered Nov 29 '16 at 10:16

So when We run the application like this over several days(or month) .there's no free of map .. when we adding more data to map for months and deleting some of them ...as you mentioned will it free that memory of deleted pairs when it needs.how this behavior affect to the performance of the application vs memory usage – Noah Nov 29 '16 at 10:33
There is no requirement that memory you just have freeed or deleted will be immediately returned to the kernel. The moment when this memory will be returned to the kernel is only known by your memory allocator(and its strategy). If you have to return memory to kernel immedeately - you have to use custom memory allocators in C++ which talks to kernel directly. And instantiate your memory with direct usage of your custom memory allocator. – Nov 29 '16 at 10:38
@Noah: When you erase the entries, the memory for those entries is released back to the heap (but not to the system). When you allocate a new entry, that memory is likely to be reused. So a long lived program should have no problems. Life is a bit more complicated because of possible heap fragmentation (if you want 1000bytes, but the heap consists of 3Gbytes worth of 999 byte holes followed by 1 byte blocks). If you need to worry about that, you may need a special purpose allocator. – Martin Bonner supports Monica Nov 29 '16 at 10:46
is 'emplace' function in c++ when we use map or list to insert new value uses 'new' or 'malloc' – Noah Nov 29 '16 at 11:44
@Noah it's not requirement that operator new is implemented by malloc function. But usually it is(at least libstdc++ and libc++). – Nov 29 '16 at 11:49
Sorry I was asking when we say use map.emplace(key,val) :to insert this new value pair into the map. what does it use inside the emplace function..is it a new or malloc.. – Noah Nov 29 '16 at 11:52
@Noah, It uses allocate function from allocator parameter for a map. And usually it uses operator new, which usually is a thin wrapper around malloc. – Nov 29 '16 at 11:59

memory usage by a process in c++ program

1 Answers1