0

If I have only the physical address of the memory buffer to which is mapped the device buffer via the PCI-Express BAR (Base Address Register), how can I map this buffer to user-space?

For example, how does usually the code should look like in Linux-kernel?

unsigned long long phys_addr = ...; // get device phys addr
unsigned long long size_buff = ...l // get device size buff

// ... mmap(), remap_pfn_range(), Or what should I do now?

On: Linux x86_64

From: https://stackoverflow.com/a/17278263/1558037

ioremap() maps a physical address into a kernel virtual address. remap_pfn_range() maps physical addresses directly to user space.

From: https://stackoverflow.com/a/9075865/1558037

int remap_pfn_range(struct vm_area_struct *vma, unsigned long virt_addr, 
    unsigned long pfn, unsigned long size, pgprot_t prot);

remap_pfn_range - remap kernel memory to userspace

May be can I use it so?

unsigned long long phys_addr = ...; // get device phys addr
unsigned long long size_buff = ...l // get device size buff

remap_pfn_range(vma, vma->vm_start, (phys_addr >> PAGE_SHIFT), 
    size_buff, vma->vm_page_prot);

Question: But, where can I get wma, and what I must pre-do with wma before call to remap_pfn_range()?

Community
  • 1
  • 1
Alex
  • 12,578
  • 15
  • 99
  • 195

2 Answers2

2

Mapping PCI resource is dependent on the architecture.

BARs are already available to userspace with the sysfs files /sys/bus/pci/devices/*/resource*, which support mmap.

This is implemented by the function pci_mmap_resource in drivers/pci/pci-sysfs.c, which ends up calling pci_mmap_page_range.

CL.
  • 173,858
  • 17
  • 217
  • 259
  • Thanks! Do I need only use `pci_mmap_resource()` to remap whole any of BARs(BAR01/23/45) from the sysfs files `/sys/bus/pci/devices/*/resource*` to userspace? Can you write a simple example of using `pci_mmap_resource()`? – Alex Nov 30 '13 at 12:46
  • 1
    Your userspace program can just open the sysfs files. These functions are not exported for drivers. – CL. Nov 30 '13 at 12:57
  • Ok. If my device has a **8GB BAR**, then at boot time in `/sys/bus/pci/devices/*/resource*` will be allocated PTEs for 8GB range? – Alex Nov 30 '13 at 13:16
  • And do I need to use `msync()` to send (flush) data to device from memory? – Alex Nov 30 '13 at 13:19
  • 1
    BARs are mapped uncached; *_wc files are mapped as write-combining. – CL. Nov 30 '13 at 14:56
1

The Linux kernel, at least, versions 2.6.x use the ioremap() function.

void *vaddr = ioremap (phys_addr, size_addr);
if (vaddr) {
  /* do stuff with the memory using vaddr pointer */
  iounmap (vaddr);
}

You should make a previous call to request_mem_region() to check if that memory space is already reclaimed by another driver, and politely request that memory to be owned by your code (driver). The complete example should look like this:

void *vaddr;
if (request_mem_region (phys_addr, size_addr, "my_driver")) {
  vaddr = ioremap (phys_addr, size_addr);
  if (vaddr) {
    /* do stuff with the memory */
    iounmap (vaddr);
  }
  release_mem_region (phys_addr, size_addr);
}

You can check your ownership by checking /proc/iomem, which will reflect the address range and the owner of every piece of memory in your system.

UPDATE: I don't really know if this works for 64-bit kernels. It does for 32-bit. If 64-bit kernel don't have these kernel functions, they will have similar ones, I guess.

mcleod_ideafix
  • 11,128
  • 2
  • 24
  • 32
  • Thanks! But should I do `ioremap_uc()` instead of `ioremap()` for PCIe-device buffer to disable CPU-cache for this region? And do `ioremap[_uc|_wc]()` functions return pointer to virtual addressing **in user-space as I need**, not in kernel-space? – Alex Nov 29 '13 at 21:02
  • About `ioremap[_uc|_wc]()` I mean these: http://stackoverflow.com/q/19811237/1558037 – Alex Nov 29 '13 at 21:06
  • 2
    This example, and these functions, only work in kernel mode. To access this memory from userspace maybe you can do it through /proc/mem or writting a device driver that implements the `mmap()` system call for your device. – mcleod_ideafix Nov 29 '13 at 21:38
  • Ok. But may be can I use directly `remap_pfn_range()` as said here? http://stackoverflow.com/a/17278263/1558037 – Alex Nov 29 '13 at 21:52
  • `memremap()` for newer kernels seems better approach. – 0andriy Feb 02 '16 at 13:13