How to intepret the virtual kernel memory layout on ARM64?

Question

The following memory layout info is printed by linux kernel at boot time, on my SoC platform (with 4-core Cortex-A53 processor and 2.75GiB DRAM), but I don't exactly understand the content of the layout:

[    0.000000] Memory: 2736316K/2883584K available (4796K kernel code, 488K rwdata, 1360K rodata, 448K init, 341K bss, 130884K reserved, 16384K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffffff8000000000 - 0xffffff8008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffffff8008000000 - 0xffffffbebfff0000   (   250 GB)
[    0.000000]       .text : 0xffffff8008080000 - 0xffffff8008530000   (  4800 KB)
[    0.000000]     .rodata : 0xffffff8008530000 - 0xffffff8008690000   (  1408 KB)
[    0.000000]       .init : 0xffffff8008690000 - 0xffffff8008700000   (   448 KB)
[    0.000000]       .data : 0xffffff8008700000 - 0xffffff800877a200   (   489 KB)
[    0.000000]        .bss : 0xffffff800877a200 - 0xffffff80087cf76c   (   342 KB)
[    0.000000]     fixed   : 0xffffffbefe7fd000 - 0xffffffbefec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffffffbefee00000 - 0xffffffbeffe00000   (    16 MB)
[    0.000000]     vmemmap : 0xffffffbf00000000 - 0xffffffc000000000   (     4 GB maximum)
[    0.000000]               0xffffffbf00000000 - 0xffffffbf02c00000   (    44 MB actual)
[    0.000000]     memory  : 0xffffffc000000000 - 0xffffffc0b0000000   (  2816 MB)

Can you please walk through those sections and/or indicate an URL for more details?

Thanks a lot!

https://unix.stackexchange.com/questions/5124/what-does-the-virtual-kernel-memory-layout-in-dmesg-imply ? — KamilCuk, Aug 20 '18 at 10:40
Thanks for your comments. I did see that post, but it's for 32-bit machine...and I guess there are some difference between 32-bit and 64-bit machines... Btw, on my system, it seems that the VA_BITS=`39`, PAGE_OFFSET=`0xffff:ffc0:0000:0000`. I don't know why in this layout, most of the ranges are below PAGE_OFFSET, and the last item is above PAGE_OFFSET... — bruin, Aug 20 '18 at 12:07
It seems I understand PAGE_OFFSET wrong (I was assuming this is the boundary between user and kernel space, but it's not, it's the half way in the kernel memory space, and there is a big hole between user space and kernel space)..."http://www.cnblogs.com/zengkefu/p/6783456.html" gives a good illustration of the memory map for Arm64 with VA_BITS 39. — bruin, Aug 20 '18 at 12:52

score 3 · Answer 1 · answered Aug 21 '18 at 01:49

After reading some posts, I come up with the following memory map diagrams.

With:

CONFIG_ARM64_PAGE_SHIFT=12
CONFIG_PGTABLE_LEVELS=3
CONFIG_ARM64_VA_BITS=39

the overall virtual memory layout:

--------------------> +--------------------------+
ffff:ffff:ffff:ffff   |                          |
                      |                          |                
                      |                          | 
ffff:ffc0:0000:0000   |----- Kernel (512G)-------| <-- PAGE_OFFSET
                      |                          |
                      |                          |
ffff:ff80:0000:0000   |                          |
--------------------> +--------------------------+
ffff:ff7f:ffff:ffff   |//////////////////////////|
                      |//////////////////////////|
                      |//////////////////////////|
                      |//////////////////////////|
                      |//////////////////////////|
                      |//////////////////////////|
                      z //////////////////////// z
                      |//////////////////////////|
                      |//////////////////////////|
                      |//////////////////////////|
                      |//////////////////////////|
                      |//////////////////////////|
0000:0080:0000:0000   |//////////////////////////|
--------------------> +--------------------------+
0000:007f:ffff:ffff   |                          |
                      |                          |
                      |                          |
                      |      User (512G)         |
                      |                          |
                      |                          |
0000:0000:0000:0000   |                          |
--------------------> +--------------------------+

...and the kernel virutal memory layout:

ffff:ffff:ffff:ffff   +--------------------------+
                      |                          |
                      |                          |
                      z                          z
                      |                          |
                      |                          |
ffff:ffc0:b000:0000   |--------------------------|                       1:b000:0000 +------------------------+
                      |                          |                                   |                        |
                      |     memory (2.75G)       |                                   | Physical DRAM (2.75G)  |
                      |                          |                                   |                        |
ffff:ffc0:0000:0000   +==========================+  <-- PAGE_OFFSET  --> 1:0000:0000 +------------------------+
                      |                          |
                      |                          |
                      |                          |
                      |      vmemmap (4G)        |
                      | +----------------------+ |
                      | |    actual (44M)      | |
ffff:ffbf:0000:0000   +--------------------------+
                      |//////////////////////////|
ffff:ffbe:ffe0:0000   +--------------------------+
                      |     PCI I/O (16M)        |
ffff:ffbe:fee0:0000   +--------------------------+
                      |//////////////////////////|
ffff:ffbe:fec0:0000   +--------------------------+
                      |     fixed (4M)           |
ffff:ffbe:fe7f:d000   +--------------------------+
                      |//////////////////////////|
ffff:ffbe:bfff:0000   +--------------------------+
                      |                          |
                      |                          |
                      |                          |
                      |                          |
                      |                          |
                      |                          |
                      z     vmalloc (250G)       z
                      |                          |
                      |                          |
                      |                          |
                      |                          |
                      |                          |
                      |                          |
ffff:ff80:087c:f76c   | +----------------------+ |
                      | |        .bss          | |
                      | |        .data         | |
                      | |        .init         | |  Kernel
                      | |        .rodata       | |
                      | |        .text         | |
ffff:ff80:0808:0000   | +----------------------+ |
ffff:ff80:0800:0000   +--------------------------+
                      |     modules (128M)       |
ffff:ff80:0000:0000   +--------------------------+

References:

How to intepret the virtual kernel memory layout on ARM64?

1 Answers1