Mapping PCIe BAR regions of size greater than 4MB in Xilinx Vivado

Question

We are developing a system with a custom processor, Microblaze and some peripherals in VC709 FPGA using Xilinx Vivado. We are using two 'PCIe : BARs' in 'AXI Bridge for PCI express gen 3'.

The changes we make in the block configuration of the PCI express block are not getting reflected on the host side.

For example, when we set the BAR0 for 2GB and BAR1 for 2MB, we get two regions of 4KB and 4MB. And this doesn't change when we tried varying different BAR combinations.

    $lspci -vv
    0a:00.0 Memory controller: Xilinx Corporation Device 7038
    Subsystem: Xilinx Corporation Device 0007
    Physical Slot: 3
    Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Interrupt: pin A routed to IRQ 16
    Region 0: Memory at fbff0000 (32-bit, non-prefetchable) [size=4K]
    Region 1: Memory at fb800000 (32-bit, non-prefetchable) [size=4M]
    Capabilities: <access denied>
    Kernel modules: riffa

How can I debug it?

You should set “prefetchable”. Most systems cannot possibly make space for a 2 GB non-prefetchable BAR, because it needs to be in the low 4 GB of the address space. See the implementation note on page 704 of the PCIe 4.0 spec. (Not sure what page of the 3.0 spec.) — prl, Dec 06 '18 at 08:30
@prl Noted. But even when we tried with 4MB and 16MB the result of were the same. — vineeshvs, Dec 06 '18 at 10:23
This *should* be based on how the device responds to a write to the BAR registers. From PCI spec: "Power-up software can determine how much address space the device requires by writing a value of all 1's to the register and then reading the value back. The device will return 0's in all don't-care address bits, effectively specifying the address space required." See also https://stackoverflow.com/a/39618552/1076479 — Gil Hamilton, Dec 07 '18 at 20:45

score 1 · Accepted Answer · answered Dec 15 '18 at 05:42

Solution

I have been using Xilinx SDK for programming the FPGA. But it the SDK was always loading an old bit stream. I have solved it by programming the FPGA through Vivado itself.

Tools > Xilinx hardware manager > open target > autoconnect > program device

Credits: Mandar Datar (HPC Lab, Dept of EE, IIT Bombay)

Mapping PCIe BAR regions of size greater than 4MB in Xilinx Vivado

1 Answers1