2

I'm developing a c++ program. This program seems to leak sometimes. Then it suddenly grows 2/3 times in size and keeps growing up to out of memory. "Run it in valgrind" you would say. I do, but the strange thing is that at exit, it shows leakage of "only" 0.5MB. Not the gigabytes I see using "top" (RSS).

The program is a program which monitors 10 webcams. They produce a constant fps, a constant resolution, rtsp or mjpeg stream. Nothing of that changes. Time periods are: days(!) of constant memory usage and then suddenly memory usage starts to grow:

example

So question now is: any tips for how to figure out these kinds of problems?

Folkert van Heusden
  • 433
  • 4
  • 17
  • 38
  • All leaks "disappear at exit" since the OS cleans up all memory the program ever allocated. – Jesper Juhl Jul 20 '20 at 20:43
  • 2
    Why do you think those are leaks and not just your software consuming that much memory? – Fureeish Jul 20 '20 at 20:43
  • 1
    @JesperJuhl yes, that's why I ran it in valgrind... – Folkert van Heusden Jul 20 '20 at 20:44
  • @Fureeish good point but its input is constant – Folkert van Heusden Jul 20 '20 at 20:44
  • "*but its input is constant*" - well, so what? A program that constantly receives 1 byte per second and keeps this data to be analyzed also has constant input, but grows infinitely in size (until out of memory occurs). No leaks there. – Fureeish Jul 20 '20 at 20:45
  • 1
    are you using unique and shared pointer when possible? – macroland Jul 20 '20 at 20:46
  • @macroland I don't. Looks useful! – Folkert van Heusden Jul 20 '20 at 20:49
  • 1
    You can have valgrind log all allocations, see https://stackoverflow.com/questions/30512000/how-to-make-valgrind-log-all-allocations. Then when you manage to trigger the large consumption, you'll be able to see what part of your code is to blame. – Nate Eldredge Jul 20 '20 at 20:54
  • 2
    Leaks that disappear at exit probably are not leaks, but rather memory resources that are held on too long (longer than needed). One mechanism I've used is have classes register with a memory monitor singleton class (I'd use a string literal with the name of the class and have that address be the key), and the string which keeps track of total number of objects allocated, current number of live objects, and highwater mark of live objects. Periodically (say once an hour for a program that runs for days), dump that information out to your log. – Eljay Jul 20 '20 at 21:05
  • How do you *expect* memory usage to grow? Does that blue line represent memory usage? It seems to jump up and then plateau for a while, which suggests that a container may have filled up, and been enlarged. If you monitor 9 cameras instead of 10, does the plateau change? Does it rise less high, or appear later? – Beta Jul 20 '20 at 21:53
  • @Beta I expect it to immediate go up to a certain level and then stay relatively steady. there should be spikes (when there are viewers) but they should go back to the base-level. – Folkert van Heusden Jul 21 '20 at 07:29

1 Answers1

1

valgrind provides various ways to track memory allocations that are later on deallocated.

To see peak of memory, you can e.g. use the valgrind massif tool.

To see which code allocates/deallocates a lot of memory, you can e.g. use memcheck option --xtree-memory=full and visualise the resulting file with kcachegrind.

phd
  • 3,669
  • 1
  • 11
  • 12