Wrong entries with /proc/pid/stat?

Question

I have written a simple C program, with some global and static variables. Some code, and output can be shown below.

Sample source code:

#include <stdio.h>

int g1, g2;

int main(){

     printf("g1:%p g2:%p\n", &g1, &g2);

     return 0;
}

Output:

g1:0x6061b0 g2:0x6061c0

Such variables falls in the data segment. They could be in the initialised subsection, or in the uninitialised (bss) subsection. I print their addresses to check where they are in memory, and then I compare these values with the values of start_data and end_data entries of the /proc/<pid>/stat pseudo-file. My variables do not fall in that area.

Limits that /proc/pid/stat gives me:

start_data: 604e10
end_data:   605180

From documentation

45) start_data %lu (since Linux 3.3) Address above which program initialized and uninitialized (BSS) data are placed.

(46) end_data %lu (since Linux 3.3) Address below which program initialized and uninitialized (BSS) data are placed.

I have managed to get the correct area of the data segment using a map file produced by the linker, however by using the proc interface it would be much cleaner (easier, faster,..) .

Correct limits I found using mapfile from linker:

start: 605168
end:   606290

I am using Ubuntu Server x64, with Linux 3.13.

Full output of stat:

29505 (myexec) R 29504 29504 1438 34822 29504 24640 52 0 0 0 0 0 0 0 20 0 1 0 55253161 4308992 24 18446744073709551615 4194304 4210644 140737488347232 140737488337560 140737348896784 0 0 0 0 0 0 0 17 0 0 0 0 0 0 6311440 6312320 6320128 140737488347816 140737488347831 140737488347831 140737488351209 0

Output of map_files:

total 0
lr-------- 1 root root 64 Apr 21 22:32 400000-41a000 -> /bin/ls
lr-------- 1 root root 64 Apr 21 22:32 619000-61a000 -> /bin/ls
lr-------- 1 root root 64 Apr 21 22:32 61a000-61b000 -> /bin/ls
lr-------- 1 root root 64 Apr 21 22:32 7ffff649e000-7ffff64a9000 -> /lib/x86_64-linux-gnu/libnss_files-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff64a9000-7ffff66a8000 -> /lib/x86_64-linux-gnu/libnss_files-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff66a8000-7ffff66a9000 -> /lib/x86_64-linux-gnu/libnss_files-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff66a9000-7ffff66aa000 -> /lib/x86_64-linux-gnu/libnss_files-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff66aa000-7ffff66b5000 -> /lib/x86_64-linux-gnu/libnss_nis-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff66b5000-7ffff68b4000 -> /lib/x86_64-linux-gnu/libnss_nis-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff68b4000-7ffff68b5000 -> /lib/x86_64-linux-gnu/libnss_nis-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff68b5000-7ffff68b6000 -> /lib/x86_64-linux-gnu/libnss_nis-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff68b6000-7ffff68cd000 -> /lib/x86_64-linux-gnu/libnsl-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff68cd000-7ffff6acc000 -> /lib/x86_64-linux-gnu/libnsl-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff6acc000-7ffff6acd000 -> /lib/x86_64-linux-gnu/libnsl-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff6acd000-7ffff6ace000 -> /lib/x86_64-linux-gnu/libnsl-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff6ad0000-7ffff6ad9000 -> /lib/x86_64-linux-gnu/libnss_compat-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff6ad9000-7ffff6cd8000 -> /lib/x86_64-linux-gnu/libnss_compat-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff6cd8000-7ffff6cd9000 -> /lib/x86_64-linux-gnu/libnss_compat-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff6cd9000-7ffff6cda000 -> /lib/x86_64-linux-gnu/libnss_compat-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff6cda000-7ffff6fa3000 -> /usr/lib/locale/locale-archive
lr-------- 1 root root 64 Apr 21 22:32 7ffff6fa3000-7ffff6fa7000 -> /lib/x86_64-linux-gnu/libattr.so.1.1.0
lr-------- 1 root root 64 Apr 21 22:32 7ffff6fa7000-7ffff71a6000 -> /lib/x86_64-linux-gnu/libattr.so.1.1.0
lr-------- 1 root root 64 Apr 21 22:32 7ffff71a6000-7ffff71a7000 -> /lib/x86_64-linux-gnu/libattr.so.1.1.0
lr-------- 1 root root 64 Apr 21 22:32 7ffff71a7000-7ffff71a8000 -> /lib/x86_64-linux-gnu/libattr.so.1.1.0
lr-------- 1 root root 64 Apr 21 22:32 7ffff71a8000-7ffff71ab000 -> /lib/x86_64-linux-gnu/libdl-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff71ab000-7ffff73aa000 -> /lib/x86_64-linux-gnu/libdl-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff73aa000-7ffff73ab000 -> /lib/x86_64-linux-gnu/libdl-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff73ab000-7ffff73ac000 -> /lib/x86_64-linux-gnu/libdl-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff73ac000-7ffff73e9000 -> /lib/x86_64-linux-gnu/libpcre.so.3.13.1
lr-------- 1 root root 64 Apr 21 22:32 7ffff73e9000-7ffff75e8000 -> /lib/x86_64-linux-gnu/libpcre.so.3.13.1
lr-------- 1 root root 64 Apr 21 22:32 7ffff75e8000-7ffff75e9000 -> /lib/x86_64-linux-gnu/libpcre.so.3.13.1
lr-------- 1 root root 64 Apr 21 22:32 7ffff75e9000-7ffff75ea000 -> /lib/x86_64-linux-gnu/libpcre.so.3.13.1
lr-------- 1 root root 64 Apr 21 22:32 7ffff75ea000-7ffff77a5000 -> /lib/x86_64-linux-gnu/libc-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff77a5000-7ffff79a4000 -> /lib/x86_64-linux-gnu/libc-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff79a4000-7ffff79a8000 -> /lib/x86_64-linux-gnu/libc-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff79a8000-7ffff79aa000 -> /lib/x86_64-linux-gnu/libc-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff79af000-7ffff79b6000 -> /lib/x86_64-linux-gnu/libacl.so.1.1.0
lr-------- 1 root root 64 Apr 21 22:32 7ffff79b6000-7ffff7bb5000 -> /lib/x86_64-linux-gnu/libacl.so.1.1.0
lr-------- 1 root root 64 Apr 21 22:32 7ffff7bb5000-7ffff7bb6000 -> /lib/x86_64-linux-gnu/libacl.so.1.1.0
lr-------- 1 root root 64 Apr 21 22:32 7ffff7bb6000-7ffff7bb7000 -> /lib/x86_64-linux-gnu/libacl.so.1.1.0
lr-------- 1 root root 64 Apr 21 22:32 7ffff7bb7000-7ffff7bd7000 -> /lib/x86_64-linux-gnu/libselinux.so.1
lr-------- 1 root root 64 Apr 21 22:32 7ffff7bd7000-7ffff7dd6000 -> /lib/x86_64-linux-gnu/libselinux.so.1
lr-------- 1 root root 64 Apr 21 22:32 7ffff7dd6000-7ffff7dd7000 -> /lib/x86_64-linux-gnu/libselinux.so.1
lr-------- 1 root root 64 Apr 21 22:32 7ffff7dd7000-7ffff7dd8000 -> /lib/x86_64-linux-gnu/libselinux.so.1
lr-------- 1 root root 64 Apr 21 22:32 7ffff7dda000-7ffff7dfd000 -> /lib/x86_64-linux-gnu/ld-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff7ffc000-7ffff7ffd000 -> /lib/x86_64-linux-gnu/ld-2.19.so
lr-------- 1 root root 64 Apr 21 22:32 7ffff7ffd000-7ffff7ffe000 -> /lib/x86_64-linux-gnu/ld-2.19.so

Any thoughts? Cheers.

procfs content changes across kernel versions. What version of kernel are you using? Are you sure you are looking at the right fields and you have kernel >=3.3 (which supports start_data and end_data)? — P.P, Apr 21 '15 at 19:17
It shows correctly (i.e. range matches with maps) for your code on my ubuntu 3.11, i686. Can you post the exact content of `stat` and `ls -l ../map_files` ? — P.P, Apr 21 '15 at 20:00
The addresses as seen inside the program are physical addresses -> virtual addresses by the kernel. Your program reports the virtual addresses. proc may be showing you what the kernel sees - the actual physical addresses. The end of memory is dynamic -- The brk system call changes the ending memory address of your resident working set size as the program runs. Try looking at a trace to see what I mean. — jim mcnamara, Apr 21 '15 at 20:07
@BlueMoon please see updated question! And keep in mind that addresses are shown in decimal! — Paschalis, Apr 21 '15 at 21:21
Yes, it's in decimal & converting it matches the map_files range. `ls` as root (sudo), ur output is not useful atm. — P.P, Apr 21 '15 at 21:28

myaut · Accepted Answer · 2015-04-21T21:47:55.257

3

Seem to be yes (or incorrect documentation).

ELF uses a small trick for specifying BSS (I cut leading zeroes):

$ objdump -x ./a.out

LOAD off    0x00000e00 vaddr 0x00600e00 paddr 0x00600e00 align 2**21
     filesz 0x00000258 memsz 0x00000268 flags rw-

For that PT_LOAD:

vaddr = 0x00600e00
vaddr + filesz = 0x00601058
vaddr + memsz = 0x00601068

So bytes [0x00600e00;0x00601058) are loaded into memory from file, while last 0x10 bytes should be present in memory too, but not loaded from ELF file -- they are zeroed because it is BSS. You can check that with objdump too:

25 .bss    00000010  00601058  00601058  00001058  2**2
           ^ size    ^ base address

However, inside binfmt loader for ELF, only vaddr + filesz is counted as end_data (see fs/binfmt_elf.c):

k = elf_ppnt->p_vaddr + elf_ppnt->p_filesz;

if (k > elf_bss)
    elf_bss = k;
if ((elf_ppnt->p_flags & PF_X) && end_code < k)
    end_code = k;
if (end_data < k)
    end_data = k;
k = elf_ppnt->p_vaddr + elf_ppnt->p_memsz;
if (k > elf_brk)
    elf_brk = k;

So:

end_data points to last byte loaded from file
elf_bss points to first byte of BSS (not exposed, used internally)
elf_brk points to last byte of BSS (not exposed, used internally)

You can check /proc/PID/maps -- it should show more reliable results.

P.S. It seems, that in-kernel documentation have been changed by random stranger: https://lkml.org/lkml/2011/12/6/604

edited Apr 21 '15 at 21:47

answered Apr 21 '15 at 21:31

myaut

11,174
2
30
62

nice answer. Thanks for taking time to explain it! However, why the `start_data` is wrong too? As for your last recommendation, I wished /proc/PID/maps was more reliable. Have a look to my yesterdays' question: http://stackoverflow.com/questions/29681712/why-global-variables-are-stored-in-heap/29683894#29683894 – Paschalis Apr 21 '15 at 21:43
1

@Paschalis, there is more than _your data_ is falling to a data segment. Check that with objdump, you can find service sections like `.got` and `.jcr`. – myaut Apr 21 '15 at 21:46
1

Based on what you suggest it seems the doc is not correct then. – P.P Apr 21 '15 at 21:47
@myaut I found these sections on a `mapfile` I produced from the `linker`. Don't know what they are though. I just verified what you said, by introducing an **initialised** variable, and it is at address 605180! – Paschalis Apr 21 '15 at 21:54
**This means that:** The area covers only **initialised data**, and **NOT** BSS. **Documentation is wrong.** – Paschalis Apr 21 '15 at 21:57
@Paschalis: you may read /proc/self/maps with that code: https://ideone.com/H6oFiQ It work correctly for me. – myaut Apr 21 '15 at 21:58
@myaut in my case, the BSS segment extends in the heap segment. Have a look here: http://stackoverflow.com/questions/29681712/why-global-variables-are-stored-in-heap/ – Paschalis Apr 21 '15 at 22:08

Wrong entries with /proc/pid/stat?

1 Answers1