5

I've got a pci device driver that currently uses dma_map_page to map a userspace address to a dma address. This is working fine but I'm trying to port this to the iommu api to get some of the benefits using groups and domains give.

Current Code: This works fine

ret = get_user_pages_fast(user_addr, one_page, flags, page);
dma_addr = dma_map_page(dev, off, size, *page, DMA_BIDIRECTIONAL);

IOMMU Code: This doesn't work

ret = get_user_pages_fast(...);
pfn = page_to_pfn(*page);
group = iommu_group_get(dev);
domain = iommu_domain_alloc(dev->bus);
iommu_attach_device(domain, dev);
iommu_attach_group(domain, group);
iommu_map(domain, iova, pfn << PAGE_SHIFT, size, IOMMU_READ|IOMMU_WRITE);

All functions return successfully, but when I pass the iova to the device the device can't use it. Has anyone worked with the iommu before and know where my problem is or where I can look? I haven't been able to find much on Linux's iommu implementation anywhere.

Edit: There were some entries in dmesg that I missed the first time around:

DEBUG: phys addr 0x7738de000
DEBUG: iova 0xdeadb000
DMAR: DRHD: handling fault status reg 2
DMAR: DMAR:[DMA Read] Request device [50:00.0] fault addr 1fdaee4000 
DMAR:[fault reason 06] PTE Read access is not set
Robert
  • 203
  • 1
  • 9
  • What is your CPU and south bridge model? Are you sure they have IOMMU? Is IOMMU enabled for your device (Intel's VT-d may be enabled only for some ports)? – osgx Jun 28 '16 at 23:33
  • The device has an iommu group assigned sometime during boot. Is there something else I need to check to make sure iommu is enabled for the device? – Robert Jun 28 '16 at 23:47
  • Robert, still, what is your HW? What about enabling pd_debug of `drivers/iommu/iommu.c` file? Why not `PFN_PHYS` but < – osgx Jun 29 '16 at 00:23
  • CPUs a Haswell Xeon, not sure about the southbridge. Do you mean pr_debug? It's enabled and I'm not getting anything in dmesg. And I didn't know about pfn_phys until just now – Robert Jun 29 '16 at 00:32
  • 2
    Do you see DMAR related messaged in dmesg ? Did you pass intel_iommu=on on the kernel command line? – Chaitanya Lala Jun 29 '16 at 01:56
  • I didn't notice those DMAR messages before. I added them to the post and looking into them myself – Robert Jun 29 '16 at 16:50

1 Answers1

0

Such operations are privileged because it is accessing page tables or may be data structures maintained inside task structure.

Please check how hypervisor do this stuff or Virtual machines handles such calls. There may be some driver interface which sets the IOMMU paging unit from Guest OS via hypervisor.

Hypervisor also executes in privileged mode.

anshkun
  • 105
  • 1
  • 12