How do I directly access reserved memory with a kernel module?

Question

I'm trying to limit the OS (Ubuntu Server 15.04) to a certain memory usage and reserve the rest but write a kernel module to read/write to the reserved memory. I figured out how to limit the usage/reserve memory using the kernel parameters "mem=4G memmap=4G@0 memmap=4G$4G" (4GB for OS and 4GB reserved, split at 4GB point) but I don't know how DMA to reserved memory works with kernel modules. I was thinking just create a proc file but I'm not sure if you can create one outside of the OS's allocated memory.

Any suggestions? Thanks!

Edit: This is for research so it doesn't need to be "nice"

Update: Maybe I don't need to write a kernel module. I just found this and I'm going to give it a shot: http://elinux.org/Memory_Management#Reserving_.28and_accessing.29_the_top_of_memory_on_startup

Update: I tried the link above but I segfault whenever I try to write. Here's my code:

    #include <fcntl.h>
    #include <stdio.h>
    #include <stdlib.h>
    #include <sys/mann.h>

    #define RESERVED_MEMORY_SIZE 0x100000000

    int main() {
            int fd;
            char *reserved_memory;

            fd = open("/dev/mem", O_RDWR | O_SYNC);
            reserved_memory = (char *) mmap(0, RESERVED_MEMORY_SIZE, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, fd, 4096);
            reserved_memory[0] = 'a';
            return 0;
    }

dmesg shows:

    a.out[1167]: segfault at ffffffffffffffff ip 00000000004005d7 sp 00007ffeffccbd80 error 7 in a.out[400000+1000]

For kicks I tried reserved_memory[1]:

    a.out[1180]: segfault at 0 ip 00000000004005db sp 00007ffc388d77b0 error 6 in a.out[400000+1000]

I'll look into the format of those messages so I can figure out what it's telling me.

Update:

I found this question by somebody with the same issue as me however the only solution appears to be a kernel rebuild. I'm going to try to avoid this so maybe my best option is a custom kernel module again. accessing mmaped /dev/mem?

Are you sure you are using `mmap` correctly? It seems that it is returning an error (-1 numeric value) as you are requesting to map a 4GiB region starting from offset 4096 and according to your question the system doesn't have 4GiB+4096Byte memory (since you splitted at 4GiB). Maybe you swapped the second and last argument? Also I don't known if /dev/mem can be used to access the whole (memory) address space or just the part occupied by usable RAM. — , Jun 30 '15 at 17:23
@knm241 I just added a check for the return value of mmap and it looks like it is failing. Good call! I guess I don't understand what the offset parameter is. I thought that is where the physical address started which would be at the 4GB point (start of reserved memory). — smbullet, Jun 30 '15 at 17:44
`/dev/mem` is a file. Reading the first byte means reading the physical address 0x0, reading the second byte means reading 0x1, reading the 100th byte means reading the address 0x63. The `offset` parameter tell `mmap` at which byte start to read the `/dev/mem`. So if you want to map the memory starting from 4GiB, the offset must be 0x100000000. The second parameter is how many bytes you want to map, this can be 4096. — , Jun 30 '15 at 17:54
Ok, that makes sense. I changed that line to `reserved_memory = (char *) mmap(0, 4096, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, fd, 0x100000000);` and it still fails to map. — smbullet, Jun 30 '15 at 18:03
Look [here](http://man7.org/linux/man-pages/man2/mmap.2.html#ERRORS) for a list of possible errors. Include the `errno.h` header and print the value of the `errno` lvalue (ie `printf("%d\n", errno)`. My guess is that you cannot map the memory the OS is not aware of. — , Jun 30 '15 at 18:07
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/82004/discussion-between-knm241-and-smbullet). — , Jun 30 '15 at 18:17

score 4 · Accepted Answer · answered Jun 30 '15 at 18:50

Ok, so I think I solved it. Turns out I just didn't understand how mmap works and I guess the kernel has no restriction on writing/reading /dev/mem if it's in reserved memory. Below are two programs that will write to my reserved spot in memory and read from it.

Write "Hello World!":

    #include <errno.h>
    #include <fcntl.h>
    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>
    #include <sys/mman.h>

    #define RESERVED_MEMORY_OFFSET  0x100000000     /* Offset is 4GB */

    int main() {
            int fd;
            char *reserved_memory;
            char *buffer = "Hello World!";

            fd = open("/dev/mem", O_RDWR | O_SYNC):
            /* Returns a pointer to the 4GB point in /dev/mem - the start of my reserved memory. Only mapping 4096 bytes. */
            reserved_memory = (char *) mmap(0, 4096, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, fd, RESERVED_MEMORY_OFFSET);
            if (reserved_memory == MAP_FAILED) {
                    printf("Failed to creating mapping.\n");
                    printf("ERRNO: %s\n", strerror(errno));
                    return -1;
            }
            sprintf(reserved_memory, "%s", buffer);
            return 0;
    }

Read from beginning of reserved memory:

    #include <errno.h>
    #include <fcntl.h>
    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>
    #include <sys/mman.h>

    #define RESERVED_MEMORY_OFFSET  0x100000000     /* Offset is 4GB */

    int main() {
            int fd;
            char *reserved_memory;
            char buffer[13];

            fd = open("/dev/mem", O_RDWR | O_SYNC):
            /* Returns a pointer to the 4GB point in /dev/mem - the start of my reserved memory. Only mapping 4096 bytes. */
            reserved_memory = (char *) mmap(0, 4096, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, fd, RESERVED_MEMORY_OFFSET);
            if (reserved_memory == MAP_FAILED) {
                    printf("Failed to creating mapping.\n");
                    printf("ERRNO: %s\n", strerror(errno));
                    return -1;
            }
            snprintf(buffer, 13, "%s", reserved_memory);
            printf("%s\n", buffer);
            return 0;
    }

Special thanks to @knm241!

How do I directly access reserved memory with a kernel module?

1 Answers1

Linked