I am unsucessfully trying to add a PCI interface (I219-V) to the OVS bridge using ovs-vsctl:
root@pc:/home/me# ovs-vsctl add-port br0 tryport -- set Interface tryport type=dpdk options:dpdk-devargs=0000:00:1f.6
ovs-vsctl: Error detected while setting up 'tryport': Error attaching device '0000:00:1f.6' to DPDK. See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is "/usr/local/var/log/openvswitch".
OVS and DPDK releases:
ovs-vsctl: 2.15.90
DB Schema 8.2.0
libdpdk: 21.02.0
The addition fails with:
2021-02-19T18:12:32.543Z|00126|dpdk|ERR|EAL: Driver cannot attach the device (0000:00:1f.6)
2021-02-19T18:12:32.543Z|00127|dpdk|ERR|EAL: Failed to attach device on primary process
2021-02-19T18:12:32.543Z|00128|netdev_dpdk|WARN|Error attaching device '0000:00:1f.6' to DPDK
2021-02-19T18:12:32.543Z|00129|netdev|WARN|tryport: could not set configuration (Invalid argument)
2021-02-19T18:12:32.543Z|00130|dpdk|ERR|Invalid port_id=32
2021-02-19T18:12:32.547Z|00131|dpdk|ERR|EAL: Driver cannot attach the device (0000:00:1f.6)
2021-02-19T18:12:32.547Z|00132|dpdk|ERR|EAL: Failed to attach device on primary process
2021-02-19T18:12:32.547Z|00133|netdev_dpdk|WARN|Error attaching device '0000:00:1f.6' to DPDK
2021-02-19T18:12:32.547Z|00134|netdev|WARN|tryport2: could not set configuration (Invalid argument)
2021-02-19T18:12:32.547Z|00135|dpdk|ERR|Invalid port_id=32
This is the content of the OVS DB, EAL seems to initialize correctly:
root@pc:/home/me# ovs-vsctl list Open_Vswitch
_uuid : 4150fd20-ecbd-4ee3-a438-0c74d0411831
bridges : []
cur_cfg : 9
datapath_types : [netdev, system]
datapaths : {}
db_version : "8.2.0"
dpdk_initialized : true
dpdk_version : "DPDK 21.02.0"
external_ids : {hostname=pc, rundir="/usr/local/var/run/openvswitch", system-id=""}
iface_types : [bareudp, dpdk, dpdkvhostuser, dpdkvhostuserclient, erspan, geneve, gre, gtpu, internal, ip6erspan, ip6gre, lisp, patch, stt, system, tap, vxlan]
manager_options : []
next_cfg : 9
other_config : {dpdk-init="true"}
ovs_version : "2.15.90"
ssl : []
statistics : {}
system_type : Ubuntu
system_version : "20.04-focal"
This is the current setup of interfaces, the device is bound to the vfio-pci using dpdk-devbind:
➜ ~ dpdk-devbind.py -s
Network devices using DPDK-compatible driver
============================================
0000:00:1f.6 'Ethernet Connection (10) I219-V 0d4f' drv=vfio-pci unused=
Network devices using kernel driver
===================================
0000:00:14.3 'Wireless-AC 9462 02f0' if=wlp0s20f3 drv=iwlwifi unused=vfio-pci *Active*
IOMMU is enabled:
➜ ~ sudo dmesg | grep -e DMAR -e IOMMU
[ 0.015758] ACPI: DMAR 0x000000006CAFE000 0000A8 (v01 LENOVO TP-N2Q 00001250 PTEC 00000002)
[ 0.055802] DMAR: IOMMU enabled
[ 0.157186] DMAR: Host address width 39
[ 0.157188] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[ 0.157196] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[ 0.157198] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[ 0.157203] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[ 0.157205] DMAR: RMRR base: 0x0000006e81a000 end: 0x0000006e839fff
[ 0.157207] DMAR: RMRR base: 0x00000079000000 end: 0x0000007d7fffff
[ 0.157210] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1
[ 0.157212] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[ 0.157213] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.159739] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 0.890022] DMAR: No ATSR found
[ 0.890108] DMAR: dmar0: Using Queued invalidation
[ 0.890113] DMAR: dmar1: Using Queued invalidation
[ 0.891306] DMAR: Intel(R) Virtualization Technology for Directed I/O
My grub and hugepages settings:
➜ ~ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.10.7-051007-generic root=UUID=455abfd8-5778-4930-8062-0ba7a48be98e ro quiet splash iommu=pt intel_iommu=on vt.handoff=7
grep HugePages_ /proc/meminfo
HugePages_Total: 2048
HugePages_Free: 1536
HugePages_Rsvd: 0
HugePages_Surp: 0
Any hints on how to debug the issue further? The EAL error report "Driver cannot attach the device" is unfortunately not very conclusive.
