0

I am looping over a series of CSV files row by row. For each row I create a BSONObjBuilder object and then add builder.obj() to a std::vector ... every 10k rows I insert the vector to the database and then clear the vector

mdb.insert(collection, myvector);
myvector.clear()

There are millions of rows in each CSV files. I have noticed that the mongod process is taking up more and more RAM.... until eventually the memory usage is at 100%

PID   USER     PR  NT VIRT   RES  SHR S %CPU %MEM
4450 mongod    15   0 73.2g  34g  33g S 45.7 97.0   3848:30 mongod

What is going on here? It seems to me like there is a memory leak? There is definitely no memory leak in my code.

Rob
  • 77
  • 2
  • 5

1 Answers1

1

When you call clear, it clears the elements, but might not reduce the capacity.
This has been answered here

Either Old skool; 

std::vector<T>(v).swap(v);

Note that C++11 has shrink_to_fit which might help, but may do nothing.

Community
  • 1
  • 1
doctorlove
  • 18,872
  • 2
  • 46
  • 62
  • I just ran my code again, this time I insert builder.obj() on every row so I don't use a vector, and I am getting the same memory leak. Do you have any other suggestions? – Rob Aug 01 '13 at 20:26
  • No sorry - just the clear on a vector was a warning to me. How many bytes are your files? How much RAM is mongodb using? – doctorlove Aug 02 '13 at 08:10
  • It seems that my issue was not a memory leak, this post seems to address the problem: http://stackoverflow.com/questions/4468873/how-to-release-the-caching-which-is-used-by-mongodb/4482465#4482465 – Rob Aug 22 '13 at 14:57