Reducing memory footprint of c++ program utilising large vectors

Question

In scaling up the problem size I'm handing to a self-coded program I started to bump into Linux's OOM killer. Both Valgrind (when ran on CPU) and cuda-memcheck (when ran on GPU) do not report any memory leaks. The memory usage keeps expanding while iterating through the inner loop, while I explicitly clear the vectors holding the biggest chunk of data at the end of the this loop. How can I ensure this memory hogging will disappear?

Checks for memory leaks were performed, all the memory leaks are fixed. Despite this, Out of Memory errors keep killing the program (via the OOM Killer). Manual monitoring of memory consumption shows an increase in memory utilisation, even after explicitly clearing the vectors containing the data.

Key to know is having three nested loops, one outer containing the sub-problems at hand. The middle loop loops over the Monte Carlo trials, with an inner loop running some sequential process required inside the trial. Pseudo-code looks as follows:

std::vector<object*> sub_problems;

sub_problems.push_back(retrieved_subproblem_from_database);

for(int sub_problem_index = 0; sub_problem_index < sub_problems.size(); ++sub_problem_index){
  std::vector< std::vector<float> > mc_results(100000, std::vector<float>(5, 0.0));
  for(int mc_trial = 0; mc_trial < 100000; ++mc_trial){
    for(int sequential_process_index = 0; sequential_process_index < 5; ++sequential_process_index){
      mc_results[mc_trial][sequential_process_index] = specific_result;
    }
  }

  sub_problems[sub_problem_index]->storeResultsInObject(mc_results);
  // Do some other things
  sub_problems[sub_problem_index]->deleteMCResults();
}

deleteMCResults looks as follows:

bool deleteMCResults() {
  for (int i = 0; i < asset_values.size(); ++i){
    object_mc_results[i].clear();
    object_mc_results[i].shrink_to_fit();
  }
  object_mc_results.clear();
  object_mc_results.shrink_to_fit();
  return true;
}

How can I ensure memory consumption to be solely dependent on the middle and inner loop instead of the outer loop? The second, and third and fourth and so, could theoretically use exactly the same memory space/addresses as utilised for the first iteration.

Answer 1

Perhaps I'm reading your pseudocode too literally, but it looks like you have two mc_results variables, one declared inside the for loop and one that deleteMCResults is accessing.

In any case, I have two suggestions for how to debug this. First, rather than letting the OOM killer strike, which takes a long time, is unpredictable, and might kill something important, use ulimit -v to put a limit on process size. Set it to something reasonable like, say, 1000000 (about 1GB) and work on keeping your process under that.

Second, start deleting or commenting out everything except the parts of the program that allocate and deallocate memory. Either you will find your culprit or you will make a program small enough to post in its entirety.

Answer 2

deleteMCResults() can be written a lot simpler.

void deleteMCResults() {
  decltype(object_mc_results) empty;
  std::swap(object_mc_results, empty);
}

But in this case, I'm wondering if you really want to release the memory. As you say, the iterations could reuse the same memory, so perhaps you should replace deleteMCResults() with returnMCResultsMemory(). Then hoist the declaration of mc_results out of the loop, and just reset its values to 5.0 after returnMCResultsMemory() returns.

Answer 3

There is one thing that could easily be improved from the code you show. However, it is really not enough and not precise enough info to make a full analysis. Extracting a relevant example ([mcve]) and perhaps asking for a review on codereview.stackexchange.com might improve the outcome.

The simple thing that could be done is to replace the inner vector of five floats with an array of five floats. Each vector consists (in typical implementations) of three pointers, to the beginnig and end of the allocated memory and another one to mark the used amount. The actual storage requires a separate allocation, which in turn incurs some overhead (and also performance overhead when accessing the data, keyword "locality of reference"). These three pointers require 24 octets on a common 64-bit machine. Compare that with five floats, those only require 20 octets. Even if those floats were padded to 24 octets, you would still benefit from eliding the separate allocation.

In order to try this out, just replace the inner vector with a std::array (https://en.cppreference.com/w/cpp/container/array). Odds are that you won't have to change much code, raw arrays, std::array and std::vector have very similar interfaces.