0

I want to implement FPGA-GPU RDMA by nvidia quadro p620.

Also, I used common PCIe BAR resources(BAR0 - BAR1 - BAR2) for FPGA registers and other chunk controllers handling which is independent from RDMA in my custom driver.

PCIe managements are OK but direct memory access to GPU ram which is pinned are always wrong. Precisely, i always get 64KB pinned addresses starting from 2955739136 (~2.7GB) by using nvidia_p2p_get_pages() API without any errors but the point is that quadro p620 ram capacity is just 2GB!.

The virtual address obtained by cuMemAlloc() change every time (which is correct) and i pass this address, together the allocated size, to my driver by ioctl sys-call. Also, i linked my custom driver to nvidia driver as the nvidia GPUDirect RDMA document is said.

Well, every things sounds OK, but the physical addresses are out of range!. Why? Does it requirement to have the qudro GPU equal or over 4GB ram address?
I expect to find the right solution to get the correct physical addresses and then DMA data by FPGA bus master.

Thanks

P.S. before this i implemented FPGA direct memory access to system ram over PCIe without any problems.

SkyCyborg
  • 1
  • 2
  • Are you sure a p620 supports bar addressing? Pascal is a very old architecture in relative terms. –  Dec 18 '22 at 09:27
  • Hi strom. No limitation is report for bar addressing by nvidia. Also, the nvidia documentation is said all qudro arch support RDMA. – SkyCyborg Dec 18 '22 at 09:31
  • RDMA is one thing, PCIe BAR is completely different it implies shared PCIe-system memory –  Dec 18 '22 at 09:36
  • That's correct. PCIe BARs accessing in my linux driver are OK. But RDMA has problem. – SkyCyborg Dec 18 '22 at 09:40

0 Answers0