I am trying to understand my embedded Linux application's memory use. The /proc/pid/maps utility/file seems to be a good resource for seeing the details. Unfortunately I don't understand all the columns and entries.
What does the anonymous inode 0 entries mean? These seem to be some of the larger memory segments.
Each row in /proc/$PID/maps describes a region of contiguous virtual memory in a process or thread. Each row has the following fields:
address perms offset dev inode pathname
08048000-08056000 r-xp 00000000 03:0c 64593 /usr/sbin/gpm
- will appear instead of the r/w/x. If a region is not shared, it is private, so a p will appear instead of an s. If the process attempts to access memory in a way that is not permitted, a segmentation fault is generated. Permissions can be changed using the mprotect system call.mmap), this is the offset in the file where the mapping begins. If the memory was not mapped from a file, it's just 0.[heap], [stack], or [vdso]. [vdso] stands for virtual dynamic shared object. It's used by system calls to switch to kernel mode. Here's a good article about it: "What is linux-gate.so.1?"You might notice a lot of anonymous regions. These are usually created by mmap but are not attached to any file. They are used for a lot of miscellaneous things like shared memory or buffers not allocated on the heap. For instance, I think the pthread library uses anonymous mapped regions as stacks for new threads.
Please check: http://man7.org/linux/man-pages/man5/proc.5.html
address perms offset dev inode pathname
00400000-00452000 r-xp 00000000 08:02 173521 /usr/bin/dbus-daemon
The address field is the address space in the process that the mapping occupies.
The perms field is a set of permissions:
r = read
w = write
x = execute
s = shared
p = private (copy on write)
The offset field is the offset into the file/whatever;
dev is the device (major:minor);
inode is the inode on that device.0 indicates that no inode is associated with the memoryregion, as would be the case with BSS (uninitialized data).
The pathname field will usually be the file that is backing the mapping. For ELF files, you can easily coordinate with the offset field by looking at the Offset field in the ELF program headers (readelf -l).
Under Linux 2.0, there is no field giving pathname.
memory mapping is not only used to map files into memory but is also a tool to request RAM from kernel. These are those inode 0 entries - your stack, heap, bss segments and more
Although the question specifically mentions embedded systems, the title only mentions proc/<pid>/maps, which is also very useful for understanding "normal" programs. In this broader context, it's important to realize that memory allocated by malloc() can end up either in the heap or in any number of anonymous memory segments. Big blocks of anonymous memory, therefore, are likely to have come from malloc().
What /proc/<pid>/maps refers to as [heap] is more precisely a contiguous region between the memory allocated for static variables (called the BSS segment) and an address called the "program break" (see diagram below). Initially, this region is empty and there is no heap. When malloc() is called, it can create/expand the heap by asking the kernel—via the brk() syscall—to move the program break. Likewise, free() can shrink the heap if all the addresses adjacent to the program break are no longer in use.
However, moving the program break is not the only way that malloc() can make more room for itself. It can also ask the kernel—via the mmap() syscall—to reserve a block of space somewhere between the stack and the heap (see diagram below). Memory allocated in this way appears in /proc/<pid>/maps as the "anonymous inode 0 entries" mentioned in the question.
It's worth elaborating on the mmap() syscall a bit. There are four kinds of memory maps that mmap() can create, and they are each used for very different purposes. First, the memory can either be tied to the contents of a certain file, or not. The latter is called an "anonymous" map. Second, the memory can either be "private" or "shared". Private means that changes made by one process will not be visible to any others; this is usually implemented in a lazy and efficient manner called "copy-on-write". Shared means that each process will get access to the same underlying physical memory. Below are the uses that I am aware of for each kind of memory map:
malloc(), BSS segments for dynamic libraries, stack space for threadsGoing back to /proc/<pid>/maps, you can figure out which kind of memory map each line describes by looking at the "pathname" and "perms" columns. (These column names come from the kernel docs). For file maps, the "pathname" column will hold an actual path to the file being mapped. For anonymous maps, the "pathname" column will be empty. There are also some special path names like [heap] and [stack]. For private and shared maps, the "perms" column will include the p or s flag, respectively.
Current implementations of malloc() use brk() for small allocations and mmap() for large ones. It makes sense to allocate small amounts of memory on the heap, because it is very often possible to find the necessary space without having to make an expensive syscall (e.g. by reusing previously freed space). However, large allocations run the risk of never being released back to the operating system. Consider what would happen if you were to make a big allocation on the heap followed by a bunch of small ones. Even if after the big allocation is freed, the program break couldn't be moved back until all the small allocations were also freed. This simple example assumes that the allocations go onto the heap in order, which is a naive approach, but it illustrates how the heap makes it much harder to free memory back to the operating system.
Here's the relevant section from man malloc:
Normally, malloc() allocates memory from the heap, and adjusts the size of the heap as required, using sbrk(2). When allocating blocks of memory larger than MMAP_THRESHOLD bytes, the glibc malloc() implementation allocates the memory as a private anonymous mapping using mmap(2). MMAP_THRESHOLD is 128 kB by default, but is adjustable using mallopt(3). Prior to Linux 4.7 allocations performed using mmap(2) were unaffected by the RLIMIT_DATA resource limit; since Linux 4.7, this limit is also enforced for allocations performed using mmap(2).
In summary, if your program uses malloc(), then malloc() is likely responsible for many of the large, anonymous segments that get mapped into virtual memory and reported by /proc/<pid>/maps.
Caveat emptor: Pretty much everything I wrote here I just learned today, so take it with a grain of salt. That said, here are links to resources I found very helpful for understanding all of this:
/proc filesystem, and unfortunately I don't think there's a way to link directly to the relevant part. But if you search for "/proc/PID/maps", you should be able to find the right place.malloc() create a heap?brk() vs mmap()malloc() and free() work