2

I am playing around with the c functions malloc and calloc and I have some questions.

I want to see if I can use these 2 functions to allocate only physical memory, my mac has 4gb or ram and when I use malloc I can allocate way more than 4gb, which means malloc allocate both physical and virtual memory.

I have a couple of questions:

  1. is there any function I can use, so that I can only allocate the physical memory (w/o allocating the virtual mem)

  2. when calling malloc and calloc and when the pointers return, is there any way I can use the pointers to determine how much physical memory are allocated and how much virtual memory are allocate?

a quick example will be really appreciated :)

thanks for your help :)

jdphenix
  • 15,022
  • 3
  • 41
  • 74
Josh
  • 692
  • 2
  • 9
  • 38

2 Answers2

3

Short answer: no and no.

Long answer: "virtual memory" doesn't mean it's stored on disk, it means its actual location is abstracted away, so your program can't tell where it's actually stored. This means the VM system can move the "same" memory around to optimize the overall memory usage of the computer. It's entirely normal for a page of virtual memory to be initially stored in RAM, then (if it isn't used for a while) paged out to disk to make room for something else, then (when it's actually accessed) paged back into RAM, then back out to disk, then back into RAM, etc.

Userspace programs always access all memory through this abstraction layer; that is, your program cannot allocate or access anything except virtual memory.

There is a class of memory called "wired". It's still virtual, but the paging policy doesn't allow it to be moved out to disk. Your program cannot allocate this; only the kernel can.

I recommend reading the Apple developer site's discussion of memory management for more details.

Gordon Davisson
  • 118,432
  • 16
  • 123
  • 151
  • 2
    Well, except that you can lock pages to RAM if you need to. – LtWorf Jan 05 '13 at 09:58
  • 1
    @LtWorf: good point; see the [mlock function](http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man2/munlock.2.html). – Gordon Davisson Jan 05 '13 at 18:17
  • I posted an answer about that, i really don't need the link. – LtWorf Jan 05 '13 at 22:42
  • @LtWorf the link was for Josh's benefit; I missed your answer that discuses it (although it also opens a can of worms about pages vs allocation blocks, zones, etc which I'm not going to try to address). – Gordon Davisson Jan 05 '13 at 23:12
  • yes I have looked at the mlock manual, however I still can't figure out the limit by using getrlimit and RLIMIT_MEMLOCK. Please see my comments above. – Josh Jan 07 '13 at 16:54
3

Yes you can use mlock() to keep an area of memory only in RAM avoiding it to be paged out. It is usually used for encryption to avoid keys to go on the disk with the danger of being retrieved by attackers afterwards.

Of course the amount of memory you can lock in RAM is limited. It will still be virtual memory but will not go to the disk, which i guess is what you want.

malloc/calloc are libc wrappers around the OS real call for allocating memory (and i have no clue of what that might be on OSX), and they keep internal buffers to avoid too frequent system calls. You should see your libc implementation of these calls and see where they keep the data and in which format, so you can access it.

LtWorf
  • 7,286
  • 6
  • 31
  • 45
  • I have tried to call mlock and munlock to lock/unlock a small amount of RAM and it worked, however when I look at the manual of mlock it says there is a limit of how much resource can be locked. I used the function getrlimit() tried to find out the max memory I can lock (RLIMIT_MEMLOCK to print out rlim_cur and rlim_max), but it prints infinity (max long long int), but if I tried to call mlock with an insane number it returns -1, which means it fails to lock. So how can I find out the limit, if I can't use getrlimit()? – Josh Jan 07 '13 at 17:31
  • 1: buf = malloc(size). 2: if(buf) then getrlimit(). 3: if(size <= limit) then mlock(buf, size). [see munlock](https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/munlock.2.html) [see getrlimit](https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/getrlimit.2.html). rlimit is only telling you the maximum possible size you can lock/unlock, you cannot lock more than you've allocated - in other words: rlimit does not tell you how much memory you can allocate. –  Oct 01 '14 at 05:12