16

I have a simple program that tries to access the physical memory in user space, where the kernel stores the 1st struct page. On a 64 bit machine this address is:

  • kernel virtual address: ffffea0000000000
  • physical address: 0000620000000000

I am trying to access this physical address through mmap in user space. But the following code crashes the kernel.

int *addr;
if ((fd = open("/dev/mem", O_RDWR|O_SYNC)) < 0 ) {
    printf("Error opening file. \n");
    close(fd);
    return (-1);
}
/* mmap.  address of first struct page for 64 bit architectures 
 * is 0x0000620000000000.
 */
addr = (int *)mmap(0, num*STRUCT_PAGE_SIZE, PROT_READ, MAP_PRIVATE,
            fd, 0x0000620000000000);
printf("addr: %p \n",addr);
printf("addr: %d \n",*addr); /* CRASH. */
Nemo
  • 2,441
  • 2
  • 29
  • 63
Vinay
  • 433
  • 1
  • 5
  • 11
  • 1
    What is the value mmap() returns in addr? – BjoernD Aug 09 '12 at 21:18
  • 1
    @BjoernD: I tried the above on a 32-bit x86 (replacing mmap offset as 0x01000000); addr = 0xffffffff . And yes, it crashes of course on the dereference. What's the solution? – kaiwan Aug 10 '12 at 06:19
  • 3
    0xffffffff == -1 -> mmap() is returning an error. According to the man page, the reason for the error is given in the 'errno' variable. So you might want to check tat. – BjoernD Aug 10 '12 at 07:43
  • Also see [How to access mmaped /dev/mem without crashing the Linux kernel?](https://stackoverflow.com/q/11891979/608639), [mmap of /dev/mem fails with invalid argument, but address is page aligned](https://stackoverflow.com/q/39134990/608639) and [How to access kernel space from user space?](https://stackoverflow.com/q/9662193/608639) – jww Jul 11 '17 at 03:24

2 Answers2

21

I think I've found the issue -- it's to do with /dev/mem memory mapping protection on the x86.

Pl refer to this LWN article: "x86: introduce /dev/mem restrictions with a config option" http://lwn.net/Articles/267427/

CONFIG_NONPROMISC_DEVMEM

Now (i tested this on a recent 3.2.21 kernel), the config option seems to be called CONFIG_STRICT_DEVMEM.

I changed my kernel config:

$ grep DEVMEM .config
# CONFIG_STRICT_DEVMEM is not set
$

When the above prg was run with the previous kernel, with CONFIG_STRICT_DEVMEM SET: dmesg shows:

[29537.565599] Program a.out tried to access /dev/mem between 1000000->1001000.
[29537.565663] a.out[13575]: segfault at ffffffff ip 080485bd sp bfb8d640 error 4 in a.out[8048000+1000]

This is because of the kernel protection..

When the kernel was rebuilt (with the CONFIG_STRICT_DEVMEM UNSET) and the above prg was run :

# ./a.out 
mmap failed: Invalid argument
#

This is because the 'offset' parameter is > 1 MB (invalid on x86) (it was 16MB).

After making the mmap offset to be within 1 MB:

# ./a.out 
addr: 0xb7758000
*addr: 138293760 
#

It works! See the above LWN article for details.

On x86 architectures with PAT support (Page Attribute Table), the kernel still prevents the mapping of DRAM regions. The reason for this as mentioned in the kernel source is:

This check is nedded to avoid cache aliasing when PAT is enabled

This check will cause a similar error to the one mentioned above. For example:

Program a.out tried to access /dev/mem between [mem 68200000-68201000].

This restriction can be removed by disabling PAT. PAT can be disabled by adding the "nopat" argument to the kernel command line at boot time.

Community
  • 1
  • 1
kaiwan
  • 2,114
  • 1
  • 18
  • 23
  • Hi Kaiwan, Thanks for pointing out the interesting config variable. 'CONFIG_STRICT_DEVMEM is not set' in my case. Kernel (3.4.6 & 3.1.0). After making the offset change the program works. But I was interested to access that address since it holds first struct page. is this possible? – Vinay Aug 10 '12 at 18:54
  • Also I set offset to 0x0000000000000000 and I get valid return address. But if I set the offset to some random address say 0x00000000000000ff, I don't get back a valid address. Do I have to set the address on page boundary? – Vinay Aug 10 '12 at 19:14
  • 1
    ARM requires using a page boundary for mmap(). From a little testing on IA32 it seems to be the case there as well...(& you've tried on the x86_64 i presume). Also, wrt your comment on accessing the first struct page, with CONFIG_STRICT_DEVMEM Off, i thought it would work (on a page boundary).. not sure about this.. – kaiwan Aug 12 '12 at 03:11
  • @Vinay, You can disable checking. For example, always make `devmem_is_allowed` return 1. I know it's a hack. ;) – chenwj Sep 06 '12 at 08:13
  • @chenwj, Thanks.. I can try that. – Vinay Sep 12 '12 at 20:46
  • @kaiwan, thanks for the detailed explanation. A small script as collection of this discussion can be find [here](https://github.com/AhmetCanSolak/physical-address-from-user-space). Please warn me if there are cavities. – trblnc Feb 16 '17 at 12:23
  • 2
    @trblnc Thanks! your implementation seems simple- to mmap a _particular_ hard-coded va. I maintain a more generalized software that can be used to gain r/w access to any memory (MMIO/registers/RAM) fr usermode; pl check out: https://github.com/kaiwan/device-memory-readwrite – kaiwan Feb 16 '17 at 12:37
  • @kaiwan, Vaow! I have to admit that your code is more complete :P, thanks a lot for sharing. – trblnc Feb 16 '17 at 15:58
3

On x86 architectures with PAT support (Page Attribute Table), the kernel can prevent the mapping of DRAM regions (even if it is compiled without setting CONFIG_NONPROMISC_DEVMEM).

The reason for this as mentioned in the kernel source is:

This check is nedded to avoid cache aliasing when PAT is enabled

This check will cause a similar error to appear in dmesg as the one mentioned in kaiwan's answer above above. For example:

Program a.out tried to access /dev/mem between [mem 68200000-68201000].

This restriction can be removed by disabling PAT.

PAT can be disabled by adding the nopat argument to the kernel command-line at boot time.

Safayet Ahmed
  • 698
  • 5
  • 14