Is there a way to know size of a pointer passed to a __free_hook in Linux?

Question

I want to track how much memory is currently allocated by a large application.

I found that I can install hooks around malloc/free/realloc in order to intercept memory allocation calls:

http://man7.org/linux/man-pages/man3/malloc_hook.3.html

So what I want to track is total bytes allocated - total bytes freed.

Now the problem is that free only takes a pointer and not a size.

In can create my own map or hashmap in my malloc hook that tracks how much memory was allocated for that pointer but that causes quite a bit of overhead.

Is there any way (even if it is a bit of a hack) to get on Linux (64 bit) the size of the ptr when free is called (with the default g++ malloc)?

possible duplicate of [Is it possible to find the Memory Allocated to the Pointer, without searching for the malloc statement](http://stackoverflow.com/questions/5813078/is-it-possible-to-find-the-memory-allocated-to-the-pointer-without-searching-fo) — Floris, Mar 13 '13 at 13:19
The question is similar but not quite the same. That person was intending to use it to figure out how long an array was in the code itself which was recommended to be avoided in those answers. In this case this question was just for tracking of total memory. — Jeroen Dirks, Mar 13 '13 at 16:21
Maybe - but the accepted answer was the same in both cases... "use malloc_usable_size()". That makes it at least a "possible duplicate" in my mind. By creating the link we provide people who stumble on one question the opportunity to see more possible answers. — Floris, Mar 13 '13 at 16:32

score 5 · Accepted Answer · edited Mar 12 '13 at 17:55

5

linux man malloc_usable_size

malloc_usable_size() returns the number of bytes available in the dynamically allocated buffer ptr, which may be greater than the requested size (but is guaranteed to be at least as large, if the request was successful). Typically, you should store the requested allocation size rather than use this function.

edited Mar 12 '13 at 17:55

Ben

1,200
7
13

answered Mar 12 '13 at 17:37

Felipe Lavratti

2,887
16
34

Oliver Charlesworth · Answer 2 · 2013-03-12T17:52:28.533

2

This isn't a direct answer to your question, but seeing as you're interested in the total allocated memory, then here's the solution:

mallinfo()

I think you'll be most interested in the uordblks field of the struct that it returns.

Note that this isn't a standard POSIX function, but I guess that's what you'd expect for non-standard introspection like this...

edited Mar 12 '13 at 17:52

answered Mar 12 '13 at 17:38

Oliver Charlesworth

267,707
33
569
680

score 1 · Answer 3 · edited May 23 '17 at 10:25

The size of the memory block is typically stored just below the pointer. While this is a hack (you said I could...), the following code runs on my Linux box:

#include <stdio.h>
#include <stdlib.h>

int main(){
int *p, n=123;
p = (int*)malloc(n*sizeof(int));
printf("allocated %d bytes for p\n", n*sizeof(int));
printf("p[-2] : %d \n", *(p-2));
printf("malloc_useable_size(p) : %d\n", malloc_usable_size(p));
free(p);
}

The output it produces is this:

allocated 492 bytes to p
p[-2] : 513
malloc_useable_size(p): 504

Note that the size in p[-2] isn't exactly 492 - there is some additional space used up due to housekeeping and boundary alignment etc.

Also note - this worked with the gcc compiler; but g++ complained about pointer conversion, and that I hadn't declared malloc_useable_size(). I added that line out of curiosity after seeing @fanl's answer. I also played around a bit with the output of mallinfo after seeing the answer of @OliCharlesworth.

You can change the value of n, and you will see that things agree very nicely - for example, if you step n (in my above code) from 100 to 119, the values of the different variables of interest are as follows:

  n | p[-2] | usable | uordblks
----+-------+--------+---------
100    417     408       416
101    417     408       416
102    417     408       416
103    433     424       432
104    433     424       432
105    433     424       432
106    433     424       432
107    449     440       448
108    449     440       448
109    449     440       448 
110    449     440       448
111    465     456       464   
112    465     456       464
113    465     456       464
114    465     456       464
115    481     472       480
116    481     472       480
117    481     472       480
118    481     472       480
119    497     488       496

There's always a difference of 9 between usable and p[-2], and of 1 between p[-2] and uordblks. The advantage of the p[-2] method is that it tells you exactly what you asked for - the size of this pointer. The other calls may actually tell you what you really wanted...

PS It's quite possible that for very large blocks of memory, you need to look at the long integer that lives at *((long int*)(p)-1). That gives me the inspiration for a nice macro:

#define PSIZE(a) (*((long int*)(a)-1))

Then you can find out the size of any pointer with

printf("my pointer size is %ld\n", PSIZE(myPointer));

Without having to worry about the type of the pointer. I confirmed that this works for different types of pointer, and for blocks > 4G. Obviously you can decide to subtract 1 in the macro so the number agrees exactly with mallinfo().

EDIT: a more complete description of what is stored just below the pointer is given in one of the answers to this earlier question. This shows that the "+1" I observed is actually due to a flag that is stored in the LSB. The correct approach is to AND the result with ~3, to clear the two LSBs, then subtract the size of the (long int*) from the result (actually the original answer subtracts 2*sizeof(unsigned long int) but I think that's wrong):

#define PSIZE(a) ((*((long int*)(a)-1))&~3 - sizeof(long int*))

The linked answer STRONGLY recommends to use this for debug only, and not to rely on it for actual code. I feel compelled to repeat that warning.

That depends heavily on how glibc's malloc du jour does things... better go with @fanl's answer. — vonbrand, Mar 12 '13 at 20:28

score 0 · Answer 4 · answered Mar 12 '13 at 17:41

You need to install a hook in the malloc to build a table of pointers caching the size of the requested block, then when you free, search for the pointer in the previous database of malloc'd items.

That way you will know how much to decrement the currently allocated heap sum (if that's your goal), and will have a handy place to list all of the "areas" of heap still being held in memory.

Is there a way to know size of a pointer passed to a __free_hook in Linux?

4 Answers4

Linked