2

I found that malloc() allocates more memory spaces than I ask for.

struct { void *ptr; int var; } msg;   // 16 bytes (checked by sizeof())

for (int i = 0; i < 100000000; i++)
    malloc(sizeof(msg));

As the aforementioned code, malloc() actually allocate 32 bytes per function call (calculated by top), but valgrind shows only 16 bytes per call indeed.

Why does malloc allocate more memory spaces than I ask for, and how to force malloc() not to waste that much memory spaces?

Surprisingly, it allocates 32 bytes also even if the struct is 24 bytes, so I guess that the memory spaces are wasted. I am not sure whether malloc() should allocate a multiple of 32 bytes. If it's true, than the memory spaces are wasted.

EDITED:

I've tested for other circumstances.

+---------+---------------------------+
|    n    | memory usage of malloc(n) |
+---------+---------------------------+
|  1 ~ 24 |         32 bytes          |
+---------+---------------------------+
| 25 ~ 40 |         48 bytes          |
+---------+---------------------------+
| 41 ~ 56 |         64 bytes          |
+---------+---------------------------+

The memory is not fully used if n is not 16 * m + 8, m ∈ ℕ. Some memory spaces wasted are understandable because of memory alignment when n equals 22, but it still should be regarded as wasted when n equals 16. On most of the platforms, the size of minimum memory access unit is 4 bytes or 8 bytes, so why does GCC implementation choose 16 bytes per increase.

Kevin Dong
  • 5,001
  • 9
  • 29
  • 62
  • 2
    There is overhead per allocation. `malloc` needs some memory for its own data structures. See for example [Malloc vs custom allocator: Malloc has a lot of overhead. Why?](http://stackoverflow.com/questions/13064850/malloc-vs-custom-allocator-malloc-has-a-lot-of-overhead-why) – kaylum Jan 31 '16 at 04:18
  • Surprisingly, it allocates 32 bytes **also** even if the struct is 24 bytes, so I guess that the memory spaces are **wasted**. – Kevin Dong Jan 31 '16 at 04:24
  • Why do you say it is wasted? Wasted implies a cost without anything in return which is not really the case here. It's the cost of managing the memory in a certain way. – kaylum Jan 31 '16 at 04:28
  • It makes no sense that malloc allocates the same size of memory spaces if the size of struct is different because the extra overhead for maintenance should be the same. – Kevin Dong Jan 31 '16 at 04:34
  • 1
    You seem to be acting as an arm chair coder hollering from the side lines. And you have made a conclusion based on two data points without really understanding the issues being solved by the implementation. Feel free to write your own *general purpose* memory allocator that is super efficient and performs better than `malloc` in all cases. You'll quickly find out that it isn't that easy. – kaylum Jan 31 '16 at 04:44
  • I agree with that, but I am trying to find the solution without writing my own memory allocator. If my problem cannot be solved easily, then I have to do that because of the low memory usage is the highest priority in this project. – Kevin Dong Jan 31 '16 at 04:50

3 Answers3

1

Any extra memory, if any, allocated by malloc(), calloc(), etc. is an implementation defined aspect of the system and not specified by C.

Check your compiler's specification to determined why. Usually it is for memory management.

To force a different allocation scheme, re-write malloc() or use your own memory allocation functions.

chux - Reinstate Monica
  • 143,097
  • 13
  • 135
  • 256
1

malloc () has very significant run-time overheads. The GNU C Library uses ptmalloc which is based on dlmalloc ("Doug Lea's Malloc").

Memory on the heap is allocated as "chunks", an 8-byte aligned data structure which contains a header, and usable memory. Allocated memory contains an 8 or 16 byte overhead for the size of the chunk and usage flags. Unallocated chunks also store pointers to other free chunks in the usable space area, making the minimum chunk size 24 bytes.

Unallocated memory is grouped into "bins" of similar sizes, implemented by using a double-linked list of chunks (with pointers stored in the unallocated space inside the chunk).

For requests below 256 bytes (a "smallbin" request), a simple two power best fit allocator is used. If there are no free blocks in that bin, a block from the next highest bin is split in two.

You can read more about it here

Madhav Datt
  • 1,065
  • 2
  • 10
  • 25
0

The excess bytes allocated i.e. the overhead is implementation specific, the memory manager in the implementation allocates as much memory as is necessary for internal tracking / house-keeping purposes. The memory space is not considered as being wasted. Your concern is genuine especially if you are looking at Millions of small allocations - this will lead to extensive heap memory fragmentation and of course the huge overhead. So I can think of two options - write your heap allocator or better use memory manager written/tested/shared by others - start with Facebook's jemalloc or Google's tcmalloc -

abRao
  • 2,787
  • 1
  • 25
  • 37