0

I have the following scenario, when starting my application, I subscribe to a middleware and download the latest state (snapshot), I need to do some filtering logic and then process the snapshot. To that aim, I put the initial snapshot in a std::unordered_map. After that I no longer need the map so I'd like its memory to be released back to the OS. The unserialized snapshot records take a considerable amount of memory, up to 15GB. Thus, it's quite important to release that memory. This is the code I have.

class Consumer
{
public:
   Consumer(Processor& p)
  : m_processor(p)
  , m_snapshot(false)
  , m_sub([this](const auto& msg) {
          if (m_snapshot)
             addToCache(msg);
          else
             m_processor.process(Order(msg));
          },
          [this](const auto& state) {
          if (state == State::BEGIN)
             m_snapshot = true;
          else if (state == State::END)
             m_snapshot = false;
             playCache();
          })
{
}

private:
   void addToCache(const Input& input)
   {
      // if logic
      m_cache[input.header.key()] = Order(input);
   }

   void playCache()
   {
       for (auto& [_, order] : m_cache)
          m_processor(std::move(order));

       m_cache.clear();
   }

private:
   Processor m_processor;
   bool m_snapshot;
   Sub m_sub;
   std::unordered_map<std::string, Order> m_cache;
};

This clearly doesn't release the memory back to the OS (checking in htop for my process it still uses the same amount of memory).

The 'm_processor' "consumes" the Order message, storing only a serialized version in a string.

I am pretty sure the memory isn't released and it's coming from this map, because I did the following tests:

  • No doing any filtering (not adding to m_cache at all) and simply processing all messages (even if they are more) yields a significantly lower memory consumption around 5GB vs 15GB of having the cache to filter and then processing.
  • I am pretty sure there's no memory leak on the upstream m_sub. I tried commenting out the part of adding to the map and processing. Aka, still getting the snapshot but throwing that away and again I get much less memory usage.

I know that .clear() doesn't imply deallocating the memory, thus as suggested in other post I tried the following:

  • Doing a swap on playCache ie:
   void playCache()
   {
       for (auto& [_, order] : m_cache)
          m_processor(std::move(order));

       m_cache.clear();
       std::unordered_map<std::string, Order> empty;
       std::swap(m_cache, empty);
   }

same result, memory is not released back to the OS.

  • Making m_cache a std::unique_ptr and calling reset once I am done:
   void playCache()
   {
       for (auto& [_, order] : *m_cache)
          m_processor(std::move(order));

       m_cache->clear();
       m_cache.reset();
   }

Still same result.

Am I missing something? Why is the memory not released back to the OS?

If it helps the data type is simply:

struct Order {
    Order() = default;
    Order(const Input& i)
    : header_(i.header())
    , body_(deserializeProto(i.body())
{
}
   protobuf::Header header_;
   protobuf::Body body_;
};

Thanks in advance.

Santiago
  • 379
  • 3
  • 14
  • 2
    `delete` is not required to return memory to OS and will rarely do so. Acquiring and releasing memory back to OS is expensive operation, so your program can keep the memory until OS requests to return it or the program ends. – Yksisarvinen Jul 27 '23 at 10:56
  • 1
    Does this answer your question? [Force memory release to the OS](https://stackoverflow.com/questions/32683699/force-memory-release-to-the-os) (spoiler alert: the answer says you can't). – Yksisarvinen Jul 27 '23 at 10:59
  • @Yksisarvinen that post speaks about virtual memory, I would expect resident memory to be returned if that memory is really released. Otherwise, scenarios like mine (needing a LOT of memory on initialization) will inevitable lead to memory leaks. – Santiago Jul 27 '23 at 11:06
  • 1
    @Santiago Why would it lead to leaks? – Ted Lyngmo Jul 27 '23 at 11:09
  • Say that your process will need in total ~1GB, however it needs to allocate 10GB during startup for whatever processing. It will never need that memory again, ever. If that memory is not given back to the OS, we will be holding 9GB of memory that the OS could allocate to other processes. – Santiago Jul 27 '23 at 11:10
  • 1
    Where do the leaks come in? – Ted Lyngmo Jul 27 '23 at 11:11
  • 1
    Resident memory and virtual memory is the same thing, in this context. Your C++ library does not release any memory back to the OS, but keeps for reuse by future allocations. If you need to release memory to the OS, in this fashion, it will become necessary to handle all the low-level allocations and deallocations yourself, and write a huge pile of code to do all of the book-keeping that this entails. – Sam Varshavchik Jul 27 '23 at 11:12
  • 1
    Keeping 15GB of memory just because it might need it later seems a bit gready for something like the C++ standard library... Anyway, this question needs a little bit more details and a full reproducable example. Otherwise there will be only speculations about what causes the isssue. – Jakob Stark Jul 27 '23 at 11:15
  • @JakobStark It's not about greediness, it's simply "I've needed this memory at one time and nobody else needs it currently, so I'll keep it in case I need it again". If OS ever requests the memory back, the program will happily give it back. If not, there's zero harm to keep it here and potential gain that you don't have to request it again. – Yksisarvinen Jul 27 '23 at 11:18
  • @Yksisarvinen How could the OS request the memory back? I am not aware of any API that would allow such a thing. – gerum Jul 27 '23 at 11:22
  • There is no mechanism for the OS to request memory back (at least not one that I am aware of in Linux). A program must return it at free will (that is call e.g. `munmap` on the mapped page), and if the C++ memory allocator keeps it, that is indeed greedy behavior. – Jakob Stark Jul 27 '23 at 11:23
  • 2
    @JakobStark OS cannot request the allocated virtual memory back from a program. But it can request its unmapping from physical memory and use the freed physical memory for a different program. – Daniel Langr Jul 27 '23 at 11:42
  • @JakobStark On windows, an application can receive a WM_COMPACTING message, which basically is a request from the os to release unused memory. – Christian Halaszovich Jul 27 '23 at 12:20
  • @ChristianHalaszovich I think that message is probably obsolete. Compacting memory is a WIN 16 thing. – Paul Sanders Jul 27 '23 at 12:40
  • @PaulSanders I agree. My point was that on some system the os can in fact ask a process to return memory. Even if most systems won't do that anymore. – Christian Halaszovich Jul 27 '23 at 13:53

0 Answers0