0

I want to allocate a fixed sized buffer in a physical memory location that would be exactly same across different runs.

void * getBuffer(size_t len, uintptr_t paddr);

So this function getBuffer(len,paddr) should allocate a buffer of len at the exact physical address paddr

So far I have seen the usage of "/dev/mem" based memory allocation that uses DMA, but this does not follow the cache access pattern according to this, this and others. (Please correct me here if I am wrong.)

But I want make sure that the memory access from the getBuffer() follows regular CPU cache patterns. i.e, the access is goes through L1, L2 and L3 cache and finally write/reads the memory.

Please let me know if this is possible. If not, is there any other way of accessing a fixed physical address? Feel free to share!

Farabi Mahmud
  • 1
  • 1
  • From userspace there is no way to do what you ask. There might be a way from kernel space, but you would have to write a custom kernel module for that. Still, you should clarify what you mean by *"this does not follow the cache access pattern"* - what cache access pattern? You mean cache *coherency* pattern? Do you want cache-coherency or not? Also, are you interested in a specific platform (in that case add the appropriate tag e.g. x86/arm/mips/...) or are you seeking a more general answer? – Marco Bonelli Apr 27 '22 at 15:17
  • See https://stackoverflow.com/questions/31140987 – Ian Abbott Apr 27 '22 at 16:10
  • See https://unix.stackexchange.com/q/37729/64699 – stark Apr 27 '22 at 16:26
  • @MarcoBonelli can you give any pointers regarding what sort of Kernel module you are referring to? I am trying to make sure that if I access an address allocated in this way, then it is following regular cache coherency. That is it goes from L1 cache, L2 cache, L3 cache and then to the actual memory. DMA bypasses cache and directly goes to memory it seems – Farabi Mahmud Apr 28 '22 at 02:46
  • @MarcoBonelli I am interested in x86 architecture. thanks for the suggestion – Farabi Mahmud Apr 28 '22 at 02:48
  • @IanAbbott I am also trying to use mmap, but does this go through cache? mmaping "/dev/mem" will be uncacheable? – Farabi Mahmud Apr 28 '22 at 02:52
  • **x86 has always had cache-coherent DMA**. The first x86 CPUs to introduce caches wanted to still be able to run OSes and other software that didn't do anything special to bypass cache, because backward-compatibility is x86's primary reason for continued existence. That means DMA had to still work with software usage patterns that didn't do anything to clear/clean caches before a DMA read or write, so hardware had to snoop any cache that existed. There was never any point where a new x86 CPU added requirements to `clflush` or anything once that instruction existed. – Peter Cordes Apr 28 '22 at 05:54
  • You tagged this [x86], but if you also care about other ISAs, then yes, on some systems software does have to make sure the destination of a DMA read is invalidated first, and that no speculative load from that area happens until the DMA read completes. – Peter Cordes Apr 28 '22 at 05:56
  • @FarabiMahmud you would simply need to write a module which allocates a piece of memory and then manages the appropriate mappings. The module will have a corresponding virtual address and r/w to/from that, which goes through CPU caches as any other memory access. The device will then need a bus address to do DMA, and as Peter says DMA on x86 already is cache coherent. Lastly, from userspace you can hse /dev/mem or expose some other kind of functionality (e.g. ioctl/proc interface/char device/etc). – Marco Bonelli Apr 28 '22 at 09:18
  • @FarabiMahmud to allocate memory you can use `kmalloc()` which gives you a kernel virtual address and then one of `dma_map_*()` which gives you the bus address. The only problem is that you want a *given* physical address. I'm not sure, but I think that cannot safely be done, unless you reserve some physical memory at boot so that the kernel doesn't use it for something else. I wouldn't know how to manage this part unfortunately. In general, you can't just safely map arbitrary physical addresses. I don't know enough to go any deeper. – Marco Bonelli Apr 28 '22 at 09:23

0 Answers0