8

I am debugging a slow memory leak in a large C++ application, I'd like to print out the current heap size at various points in the program.

Aside from opening and parsing /proc/PID/statm is there any library API call from which I can get this information?

One post suggested sbrk(), however, that returns the current heap pointer – not 100% what I want. (second question: do changes in the sbrk() value correspond to changes in the current heap size?)

I've looked but it seems strange that there'd be no system call...

Thanks

UPDATE I

I've done some test comparision between calling sbrk() and reading proc/.../statm. It appears that sbrk() does not reflect actual allocations. Instead it appears that statm measures actual allocations while sbrk() shows the total heap size.

This total heapsize increments in big chunks (equal to page size?).

The test program below produces this output (Heap Size as reported by sbrk() and memory usage as reported by /proc/.../statm, clearly showing a difference:

0 ALLOC: HEAP SIZE: 0
MEMORY USAGE: 1308 201 174 2 0 566 0
1 ALLOC: HEAP SIZE: 135168
MEMORY USAGE: 1565 212 184 2 0 823 0
2 ALLOC: HEAP SIZE: 135168
MEMORY USAGE: 1822 216 187 2 0 1080 0
3 ALLOC: HEAP SIZE: 135168
MEMORY USAGE: 2079 217 187 2 0 1337 0
4 ALLOC: HEAP SIZE: 135168
MEMORY USAGE: 2336 218 187 2 0 1594 0
5 ALLOC: HEAP SIZE: 135168
MEMORY USAGE: 2593 219 187 2 0 1851 0

0 FREE: HEAP SIZE: 135168
MEMORY USAGE: 3364 225 189 2 0 2622 0
1 FREE: HEAP SIZE: 135168
MEMORY USAGE: 3107 224 189 2 0 2365 0
2 FREE: HEAP SIZE: 135168
MEMORY USAGE: 2850 223 189 2 0 2108 0
3 FREE: HEAP SIZE: 135168
MEMORY USAGE: 2593 222 189 2 0 1851 0
4 FREE: HEAP SIZE: 135168
MEMORY USAGE: 2336 221 189 2 0 1594 0
5 FREE: HEAP SIZE: 135168
MEMORY USAGE: 2079 220 189 2 0 1337 0

Test Program

class CfgProfileList
{
public:
    bool obtainSystemProfileList();
    void leakObjTest();
    std::set<std::string> mProfileList;
private:
    char dummy[1024 * 1024]; // use up some space
};

class ComUtil
{
public:
    static void printMemoryUsage();
private:
    static unsigned int mHeapOrigin;
};

/* static */
unsigned int ComUtil::mHeapOrigin = 0;

// Print current process memory utilization
/* static */ void
ComUtil::printMemoryUsage()
{
    unsigned int pHeap = (unsigned int)sbrk(0);
    if (mHeapOrigin == 0)
        mHeapOrigin = pHeap;

    printf("HEAP SIZE: %u\n", pHeap - mHeapOrigin);

    char fname[256], line[256];
    sprintf(fname, "/proc/%d/statm", getpid());

    FILE *pFile = fopen(fname, "r");
    if (!pFile)
        return;    
    fgets(line, 255, pFile);
    fclose(pFile);
    printf("MEMORY USAGE: %s", line);
}   

void
CfgProfileList::leakObjTest()
{
    CfgProfileList *pointerList[50];
    int  n = 10;
    int  sleep = 100000;

    printf("OBJECT ALLOCATION\n");    
    for (int i = 0; i < n; i++)
    {
        pointerList[i] = new CfgProfileList;
        printf("%d ALLOC: ", i);
        ComUtil::printMemoryUsage();
        usleep(sleep);
    }

    printf("\n");

    for (int i = 0; i < n; i++)
    {
        delete pointerList[i];
        printf("%d FREE: ", i);
        ComUtil::printMemoryUsage();
        usleep(sleep);
    }
}

int
main(int argc, char **argv)
{
    CfgProfileList pl;
    pl.leakObjTest();
}
Danny
  • 2,482
  • 3
  • 34
  • 48
  • Have you considered using valgrind or similar? – paddy Dec 12 '16 at 05:13
  • Unfortunately, Valgrind doesn't work; dies with illegal instruction bytes detected. I've asked directly on the valgrind developer list and, following their suggestions, I have since a) rebuild valgrind from source and b) recompiled gcc to the native processor of the target machine. Doesn't work. Can see related question here: http://stackoverflow.com/questions/41088970/reliability-of-proc-statm-for-finding-memory-leak – Danny Dec 12 '16 at 07:12
  • 1
    What is your problem? You have illegal instruction? Maybe a pointer of function invalid? Please add what valgrind print. Seems like a XY problem – Stargateur Dec 12 '16 at 07:26
  • 1
    You may use custom allocators( you may need to write a simple one if you're not worried about speed) to check the memory leak. Some place you may also use placement new operator (which allocate memory from a already allocated memory) – Daksh Gupta Dec 12 '16 at 10:41
  • Stargateur, you can see the full thread on the valgrind list here: https://sourceforge.net/p/valgrind/mailman/valgrind-users/thread/E0EA7B0B-FEFC-4F3B-9BE7-0538F4434653%40torquevideo.tv/#msg35516631 I've given up on valgrind for this project. I have been having some success with gperftools! – Danny Dec 13 '16 at 01:14
  • The fact that `valgrind` does not work should go into the question. It is very relevant – Basile Starynkevitch Aug 28 '18 at 08:30

2 Answers2

11

Since glibc's new is based on malloc(), the malloc information and debugging functions can be used; for example you could add a call of malloc_stats() to your application.

#include <malloc.h>
…
    malloc_stats();

The malloc_stats() function prints (on standard error) statistics about memory allocated by malloc(3) and related functions. …

You might also have a look at

  • mallinfo(),

    The mallinfo() function returns a copy of a structure containing information about memory allocations performed by malloc(3) and related functions. …

  • malloc_hook,

    The GNU C library lets you modify the behavior of malloc(3), realloc(3), and free(3) by specifying appropriate hook functions. You can use these hooks to help you debug programs that use dynamic memory allocation, for example.

  • and mtrace().

    The mtrace() function installs hook functions for the memory-allocation functions (malloc(3), realloc(3) memalign(3), free(3)). These hook functions record tracing information about memory allocation and deallocation. The tracing information can be used to discover memory leaks and attempts to free nonallocated memory in a program.

Armali
  • 18,255
  • 14
  • 57
  • 171
2

I am debugging a slow memory leak in a large C++ application

You should try using valgrind (after compiling all your code with -g). It is a very good tool to track memory leaks (but it does slow down your program by a significant factor, perhaps x3 or x10 at least).

I'd like to print out the current heap size at various points in the program.

I am not sure that heap size has a very well defined meaning (but see other answers to your question). The OS provides and manages the virtual address space of your process, and some parts of that are conventionally called "heap". You could use proc(5) and parse (from inside your program) /proc/self/stat, /proc/self/statm, /proc/self/status, /proc/self/maps, and this can be quick. Also use the pmap(1), ps(1), top(1) commands (all using /proc/ internally). The mmap(2) & munmap, mprotect(2) (and old sbrk(2)) system calls can change your virtual address space. the C++ new uses malloc which often use mmap. However, free or delete often mark a released memory zone as reusable by future malloc-s but don't give up the memory to the system.

PS. If valgrind is not working, my guess is that you have corrupted your memory, which is even more bad than just a memory leak. Consider trying some sanitizer (and other instrumentation), perhaps the address sanitizer. Of course enable all warnings and debug info (so compile with g++ -Wall -Wextra -g) and take time to improve the source code to get no warnings at all. Recent GCC compilers (so GCC 8 in august 2018) gives better warnings and have improved sanitizrs than older ones, so it could be worth to update your g++ compiler.

Basile Starynkevitch
  • 223,805
  • 18
  • 296
  • 547