How to implement Dynamically Growing Heap

Question

Finished implementing the minimal Heap structure with awesome help from Micheal . Right now I have one thing missing so it dynamically grows. As you can see in this image, it isn't dynamic:

Instead of having a fixed size when I do pmm_map_physical_virtual(0x800000,0xC0800000), the max I can go to is 8MB which is from 0xC00000000-0xC0800000 ... but then I have to be aware not to overwrite the kernel, so I will technically have less than 8MB . Is there a way so that the heap just continues from the end of the kernel and grows dynamically? They said this is what I was missing from my memory management. And is this a good idea to do this?

Modify your heap routines so that when it can't fulfill a request (because it doesn't have room), you add a new block to the heap and perform any necessary memory mappings (page entries) to fulfill that request. Which also means you should have some way of keeping track of the free physical page frames that can be used. Your current PMM doesn't have that capability (which it should) — Michael Petch, Aug 25 '16 at 19:30
@MichaelPetch ok so the goal here is first to make the heap avialable space the whole ram . And then add blocks whenever he finishes the block? If so, how do i make heap reserve the whole space.. by that i mean you can see ``pmm.map_physical_virtual(0x400000,0xC0400000);`` , reserves 4MB.. how do i translate this physical address 0xFFFFFFFF to virtual address.. So i reserve the entire memory for the heap — amanuel2, Aug 25 '16 at 20:02
@MichaelPetch ok im about to go to sleep , can you give me little explanation why its not giving me a page fault, when i allocate 0 bytes , but write data to it? Makes no sense why its not giving me a page fault : https://github.com/amanuel2/OS_Mirror/blob/master/kernel.c%2B%2B#L90 . Ok im out — amanuel2, Aug 26 '16 at 04:26
A page fault occurs only when you write to a region of memory that hasn't been mapped or is not present. Writing to such memory will cause a page fault exception. Your `malloc` implementation returns a pointer when passed size zero. This is [allowed by C/C++](http://stackoverflow.com/questions/1073157/zero-size-malloc) . Writing to the returned address works because the address is in a valid and present memory page. The problem is that writing to that memory could conceivably corrupt the heap and may eventually lead to a page fault. — Michael Petch, Aug 26 '16 at 12:37
@MichaelPetch oh ok.. well no one is stupid enough to malloc(0) unless they are testing. Well, question i asked you above, do you know what 0xFFFFFFFF translates to virtual memory? "ok so the goal here is first to make the heap avialable space the whole ram . And then add blocks whenever he finishes the block? If so, how do i make heap reserve the whole space.. by that i mean you can see pmm.map_physical_virtual(0x400000,0xC0400000); , reserves 4MB.. how do i translate this physical address 0xFFFFFFFF to virtual address.. So i reserve the entire memory for the heap" — amanuel2, Aug 26 '16 at 16:13
Shift 0xFFFFFFFF right 22 bits and you get the BootPageDirectory index of 0x3FF (1023 decimal) to map to a physical memory region. You can also take 0xFFFFFFFF and round it down to the nearest 4MB by bitwise ANDing with 0xFFC00000 (-4MB) which gives you 0xFFC00000 — Michael Petch, Aug 26 '16 at 16:58
@MichaelPetch So its : `BootPageDirectory[1023]=0xFFFFFFFF invalidate_page_vm((void *)0x3FF);` to map the entire usable memory to Heap? — amanuel2, Aug 26 '16 at 17:01
No, not at all. You already wrote a function that takes a Virtual Address and a Physical Address already and updates the page directory. — Michael Petch, Aug 26 '16 at 17:03
In general your question makes no sense (or is ambiguous). The question I am referring to is " Well, question i asked you above, do you know what 0xFFFFFFFF translates to virtual memory?" . I'm probably trying to make sense of a concept you don't understand. I recommend you ask these questions on #osdev — Michael Petch, Aug 26 '16 at 17:06
Have you considered going with a flat memory model without paging that goes from 0x00000000 to 0xFFFFFFFF and doesn't use a higher half kernel? Just to start until you understand paging? You might even consider a lower half kernel with paging and then you can just identity map all 1024 4MB(1024*4MB=4GB) page directory entries. I think you have bitten off too many concepts that you don't fully understand and have made things difficult for yourself. — Michael Petch, Aug 26 '16 at 17:16
@MichaelPetch Ok ... let me ask you my question in a diffrent way.. How do i get the virtual address from a physical address.. for example to get the page number you shift right 22. — amanuel2, Aug 26 '16 at 18:06
You don't get a virtual address from a physical address (it is in fact possible to map many virtual pages to the same physical page frame. ie: I could in theory map all 4MB pages to the first 4MB of memory - not very useful but can be done by storing 0x00000083 in all 1024 BootPageDirectory entries. — Michael Petch, Aug 26 '16 at 18:13
What you are really missing is a proper PhysicalMemoryManager that determines at start up all the 4MB page frames in Physical memory that were marked "Available"(usable) by GRUB. Once you have a list of the start address of each of those 4MB page frames you can then serve them out as needed and map them into Virtual Address space.And if you want you could rewrite things to use 4KB page frames, but then you need to concern yourself with allocating page tables and page directories. That is trivial to do, but probably not for a beginner with limited knowledge of paging on x86. — Michael Petch, Aug 26 '16 at 18:17
@MichaelPetch Im not gonna distrub others and just ask you here, since your really the only legitmate person to anwser... Well the question is kinda awkward.. The performance becomes slow in VirtualBox/VMWare which runs on ISO... But becomes good with Qemu which runs with BIN.. ISO Slow : https://www.youtube.com/watch?v=hfqOIYCDods&feature=youtu.be Fast Bin : https://www.youtube.com/watch?v=LtOHpOa7Y5I .... Any Reasons for this? Btw yes i implemented the Redraw() : https://github.com/amanuel2/OS_Mirror/blob/master/desktop.c%2B%2B#L23 ... Any Help would be greatly appreciated! — amanuel2, Sep 03 '16 at 02:47
@MichaelPetch Continuing from Above.. Is this a performance problem? Do i need to figure out to make this faster in terms of performance.. And why does it become fast in BIN? ... Im thinking some how some way get the pixel you are left and somehow just redraw that pixel.. — amanuel2, Sep 03 '16 at 02:48

How to implement Dynamically Growing Heap

0 Answers0

Linked