0

I want to set up some per-user memory limits.conf on a Linux system, and for the sake of testing I wrote a minimal program:

#include <stdio.h>
#include <stdlib.h>

int main()
{
        size_t mbs = 32;
        char * leak;
        while (1)
        {
//              leak = calloc(1024 * 1024 * mbs, sizeof(char));
                leak = malloc(1024 * 1024 * mbs * sizeof(char));
                printf("%9lu MiB allocated at address %p.", mbs, leak);
//              printf(" Last byte has value %d.", leak[1024 * 1024 * mbs - 1]);
                printf(" Press enter to double them.");
                getchar();
                mbs *= 2;
        }
}

Execution is kinda

$ gcc -g -O0 memleak.c -o memleak  && ./memleak
       32 MiB allocated at address 0x14b92f312010. Press enter to double them.
       64 MiB allocated at address 0x14b92b311010. Press enter to double them.
<.....>
    32768 MiB allocated at address 0x14a933308010. Press enter to double them.
    65536 MiB allocated at address 0x149933307010. Press enter to double them.
   131072 MiB allocated at address (nil). Press enter to double them.
   262144 MiB allocated at address (nil). Press enter to double them.^C

On a system equipped with 64 GiB of RAM and 2 GiB of swap space. You can find out that mallocs start to fail after allocating more than 65536 MiB.

What puzzles me is that it does not seems the leak is something "real". I mean, on another shell I'm monitoring the process with

$ top -c -p $(pgrep memleak)
    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                                                    
3102203 pbertoni  30  10   35268  33772   1012 S   0,0   0,1   0:00.01 ./memleak  

And only VIRT is actually following the geometric series. RES stays constant: with calloc it's around the first call, ~32 MiB. With malloc, it goes down to 580 bytes.

The dereference array-like operation throws a SIGSEGV when crossing the physical threshold of 64 GiB, that's why it's being commented.

Anyway, my point is: why am I taking "virtual" memory when I actually write it (with calloc for example)? I was expecting to take "resident" memory instead. What am I missing?

Patrizio Bertoni
  • 2,582
  • 31
  • 43
  • 3
    `calloc()` doesn't necessarily write to the memory it allocates. If it does an anonymous mapping, the page table entries will be copy-on-write references to some zeroed memory. – Ian Abbott Nov 10 '21 at 17:53
  • See also [Why is malloc not "using up" the memory on my computer?](https://stackoverflow.com/q/19991623/2410359). – chux - Reinstate Monica Nov 10 '21 at 18:57

1 Answers1

1

The short answer is that the "resident" memory ranges are a subset of the "virtual" memory ranges used by a process and that you can't count on pages that are currently resident staying resident because if your process doesn't access a given range for a bit, the operating system is free to make some or part of it non-resident so that the corresponding physical memory can be used for something else. For example, in your particular program, at the point it calls getchar(), which may take arbitrarily long given that it is waiting for user input, your pages may become non-resident at some point during that call.

Consider the output of top from your question:

$ top -c -p $(pgrep memleak)
    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                                                    
3102203 pbertoni  30  10   35268  33772   1012 S   0,0   0,1   0:00.01 ./memleak  

Notice the columns for "VIRT" and "RES". The number for "VIRT" in this particular case is only slightly more than RES, meaning that all but roughly 2 MiB (35268KB - 33772KB) of the virtual address space used by the process was resident at the time you ran top. At a guess, you ran top at the point where your program was at the call to getchar() on the first time through the loop, but not so late that the given range got pushed out of memory by the time you ran top.

If you want to answer the question of which part of the virtual address space of your process is non-resident, again keeping in mind that this is subject to change based on other demands for memory on your server, you can check this by starting your program and letting it reach the first getchar() call, finding out the pid (for example by using "ps -fe" from a different session, then doing the following:

cat /proc/pid-of-your-process/smaps >mysmaps

Now edit mysmaps and look for places where the "size" (which is the virtual address size for a range) is different from the "res"

So, for example, if you leave your program at that first getchar() call (by not providing user input) and run some other programs that also use memory you will see that your pages will get pushed out of memory and you may see a section like the following in the mysmaps file you create at that time:

7f991fd23000-7f9921d24000 rw-p 00000000 00:00 0
Size:              32772 kB
Rss:                   4 kB
Pss:                   4 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:         4 kB
Referenced:            4 kB
Anonymous:             4 kB
AnonHugePages:         0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB
ProtectionKey:         0
VmFlags: rd wr mr mp me ac sd

What you can see from that is that at this point your 32MB allocation is no longer resident, because activity by other programs has pushed it out of memory.

Tim Boddy
  • 1,019
  • 7
  • 13
  • Ok it's getting more clear now. What I'm still missing is "where" the memory goes if not resident. I mean, it works only by mean of a swap partition, right? I have to try it after swapping off. – Patrizio Bertoni Nov 15 '21 at 18:41
  • Non-resident memory certainly includes swapped out memory but it certainly includes other kinds of memory. For example: – Tim Boddy Nov 15 '21 at 19:39
  • 1
    Non-resident memory certainly includes swapped out memory, and that is what happened in the example smaps output I showed where the dynamically allocated range was no longer in memory, but non-resident certainly can include other kinds of memory. For example, memory used to hold executable code is generally considered to be "file backed" in that an image of the code is present in an executable or shared library, so the operating system is free to reuse physical memory that contains an image of this code because it can always read it in again from the executable or binary. – Tim Boddy Nov 15 '21 at 19:46