0

I implementing the Poll Mode Driver(PMD) memif(vdev=net_memif) virtual device in application. Specificly, I have to run memif as a client in secondary process.According to the Dpdk pmd multiprocess application specification, I have to initialize memif vdev at primary process and use the device at secondary process. With this information I can implement the server memif vdev which can run at secondary process.The steps that I have done to make it work like below

  1. For primary process I passed this argument to rte_eal_init(): ./primaryApp --proc-type=primary --file-prefix=pmd1 --vdev=net_memif,role=server,socket=/tmp/memif.sock, socket-abstract=no
  2. then I passed this argument to secondary process rte_eal_init(): ./secondaryApp --proc-type=auto --file-prefix=pmd1.
  3. I initialized memif eth device, configured it,then started the eth device.

But in opposite configuration (run memif as client at secondary process), I cannot able to run by implementing the same steps.

I have a question want to ask.

  1. Do I have to initialize, configure and run memif client eth device on primary process to be able to run memif client on secondary process?
  2. If client memif eth device can be run as same as the server, what am I missing ?

This is the configuration step for memif eth device which runs at secondary process,

        if(!rte_eth_dev_get_port_by_name(m_name.c_str(),&m_portId))
            std::cout << "Get net_memif eth device successfully port id : " << m_portId << "\n";
        else
        {
            std::cout << "rte_eth_dev_get_port_by_name ERROR \n";
            return false;
        }

        auto eth_count = rte_eth_dev_count_avail();

        if(eth_count < 0)
            return false;


        if(!rte_eth_dev_info_get(m_portId,&ethInfo)) {

            std::cout << "Get eth dev info successfully\n";
            std::cout << " port id : " << m_portId <<"ethInfo : " << ethInfo.driver_name << "\n";
        }
        else
        {
            std::cout << "Get eth dev info ERROR\n";
            return false;
        }


        if(!rte_eth_dev_configure(m_portId, m_config.queue_rx_count, m_config.queue_tx_count, &ethConfig)) {

            std::cout << "Configure eth dev successfully\n";
        }
        else
        {
            std::cout << "Configure eth dev ERROR\n";
            return false;
        }

        if(!rte_eth_dev_adjust_nb_rx_tx_desc(m_portId, &m_config.queue_rx_desc, &m_config.queue_tx_desc))
            std::cout << "Adjust eth dev rx tx desc successfully\n";
        else
        {
            std::cout << "Adjust eth dev rx tx desc ERROR\n";
            return false;
        }

        for (int i = 0; i < m_config.queue_rx_count; i++) {
            if(!rte_eth_rx_queue_setup(m_portId, i, m_config.queue_rx_desc, rte_eth_dev_socket_id(m_portId), &ethInfo.default_rxconf, mempool))
                std::cout << "Setup rx queue successfully socket id : " << rte_eth_dev_socket_id(m_portId) << std::endl;
            else
            {
                std::cout << "Setup rx queue ERROR\n";
                return false;
            }
        }


        for (int i = 0; i < m_config.queue_tx_count; i++) {
            if(!rte_eth_tx_queue_setup(m_portId, i, m_config.queue_tx_desc, rte_eth_dev_socket_id(m_portId), &ethInfo.default_txconf))
                std::cout << "Setup tx queue successfully socket id : " << rte_eth_dev_socket_id(m_portId) << "\n";
            else
            {
                std::cout << "Setup tx queue ERROR\n";
                return false;
            }
        }

then I am starting the memif eth device as fallows,

        if( rte_eth_promiscuous_enable(m_portId) != 0) {
            std::cout << "rte_eth_promiscuous_enable error\n";
            return false;
        }
        else
            std::cout << "rte_eth_promiscuous_enable success \n";


        if(!rte_eth_dev_start(m_portId)) {
            std::cout << "Eth dev is started \n";
        }
        else {
            std::cout << "Eth dev is start ERROR \n";
            return false;
        }

Can anybody give me a clue about this issue ? I could not find an information such an implementation and dpdk documentations are not well documented. Any help will be appreciated. The primary process detailed output (it just run rte_eal_init()):

sudo bin/test-memif-dummy --proc-type=primary --file-prefix=pmd2 --vdev=net_memif0,id=0,role=client,socket=/tmp/memif_test.sock,socket-abstract=no --socket-mem=512 --legacy-mem --log-level=eal,8
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 0 on socket 0
EAL: Detected lcore 5 as core 1 on socket 0
EAL: Detected lcore 6 as core 2 on socket 0
EAL: Detected lcore 7 as core 3 on socket 0
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 8 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Checking presence of .so 'librte_eal.so.21.0'
EAL: Checking presence of .so 'librte_eal.so.21'
EAL: Checking presence of .so 'librte_eal.so'
EAL: Detected static linkage of DPDK
EAL: Ask a virtual area of 0x7000 bytes
EAL: Virtual area found at 0x100000000 (size = 0x7000)
EAL: Multi-process socket /var/run/dpdk/pmd2/mp_socket
EAL: DPAA Bus not present. Skipping.
EAL: VFIO PCI modules not loaded
EAL: Bus dpaa_bus wants IOVA as 'DC'
EAL: Bus fslmc wants IOVA as 'DC'
EAL: Bus pci wants IOVA as 'DC'
EAL: Buses did not request a specific IOVA mode.
EAL: Module /sys/module/rte_kni not found! error 2 (No such file or directory)
EAL: IOMMU is not available, selecting IOVA as PA mode.
EAL: Selected IOVA mode 'PA'
EAL: No available hugepages reported in hugepages-2048kB
EAL: Probing VFIO support...
EAL:   IOMMU type 1 (Type 1) is supported
EAL:   IOMMU type 7 (sPAPR) is not supported
EAL:   IOMMU type 8 (No-IOMMU) is not supported
EAL: VFIO support initialized
EAL: Ask a virtual area of 0x2e000 bytes
EAL: Virtual area found at 0x100007000 (size = 0x2e000)
EAL: Setting up physically contiguous memory...
EAL: Setting maximum number of open files to 1048576
EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
EAL: Detected memory type: socket_id:0 hugepage_sz:2097152
EAL: Creating 2 segment lists: n_segs:32 socket_id:0 hugepage_sz:1073741824
EAL: Ask a virtual area of 0x1000 bytes
EAL: Virtual area found at 0x100035000 (size = 0x1000)
EAL: Memseg list allocated at socket 0, page size 0x100000kB
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x140000000 (size = 0x800000000)
EAL: VA reserved for memseg list at 0x140000000, size 800000000
EAL: Ask a virtual area of 0x1000 bytes
EAL: Virtual area found at 0x940000000 (size = 0x1000)
EAL: Memseg list allocated at socket 0, page size 0x100000kB
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x980000000 (size = 0x800000000)
EAL: VA reserved for memseg list at 0x980000000, size 800000000
EAL: Creating 4 segment lists: n_segs:8192 socket_id:0 hugepage_sz:2097152
EAL: Ask a virtual area of 0x61000 bytes
EAL: Virtual area found at 0x1180000000 (size = 0x61000)
EAL: Memseg list allocated at socket 0, page size 0x800kB
EAL: Ask a virtual area of 0x400000000 bytes
EAL: Virtual area found at 0x1180200000 (size = 0x400000000)
EAL: VA reserved for memseg list at 0x1180200000, size 400000000
EAL: Ask a virtual area of 0x61000 bytes
EAL: Virtual area found at 0x1580200000 (size = 0x61000)
EAL: Memseg list allocated at socket 0, page size 0x800kB
EAL: Ask a virtual area of 0x400000000 bytes
EAL: Virtual area found at 0x1580400000 (size = 0x400000000)
EAL: VA reserved for memseg list at 0x1580400000, size 400000000
EAL: Ask a virtual area of 0x61000 bytes
EAL: Virtual area found at 0x1980400000 (size = 0x61000)
EAL: Memseg list allocated at socket 0, page size 0x800kB
EAL: Ask a virtual area of 0x400000000 bytes
EAL: Virtual area found at 0x1980600000 (size = 0x400000000)
EAL: VA reserved for memseg list at 0x1980600000, size 400000000
EAL: Ask a virtual area of 0x61000 bytes
EAL: Virtual area found at 0x1d80600000 (size = 0x61000)
EAL: Memseg list allocated at socket 0, page size 0x800kB
EAL: Ask a virtual area of 0x400000000 bytes
EAL: Virtual area found at 0x1d80800000 (size = 0x400000000)
EAL: VA reserved for memseg list at 0x1d80800000, size 400000000
EAL: Trying to obtain current memory policy.
EAL: Setting policy MPOL_PREFERRED for socket 0
EAL: Setting policy MPOL_PREFERRED for socket 0
EAL: Restoring previous memory policy: 0
EAL: Hugepage /mnt/hugepages/pmd2map_1 is on socket 0
EAL: Hugepage /mnt/hugepages/pmd2map_0 is on socket 0
EAL: Requesting 1 pages of size 1024MB from socket 0
EAL: Attempting to map 1024M on socket 0
EAL: Allocated 1024M on socket 0
EAL: Added 1024M to heap on socket 0
EAL: TSC frequency is ~2000000 KHz
EAL: Main lcore 0 is ready (tid=7f12244c1c00;cpuset=[0])
EAL: lcore 1 is ready (tid=7f12234ba400;cpuset=[1])
EAL: lcore 2 is ready (tid=7f1222cb9400;cpuset=[2])
EAL: lcore 3 is ready (tid=7f12224b8400;cpuset=[3])
EAL: lcore 4 is ready (tid=7f1221cb7400;cpuset=[4])
EAL: lcore 5 is ready (tid=7f12214b6400;cpuset=[5])
EAL: lcore 6 is ready (tid=7f1220cb5400;cpuset=[6])
EAL: lcore 7 is ready (tid=7f1203fff400;cpuset=[7])
EAL: PCI device 0000:00:1f.6 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 8086:15d8 net_e1000_em
EAL:   Not managed by a supported kernel driver, skipped
EAL: No legacy callbacks, legacy socket not created
Application is started 
EAL: msg: bus_vdev_mp
EAL: sendmsg: bus_vdev_mp
EAL: reply: bus_vdev_mp
EAL: msg: eal_vfio_mp_sync
EAL: reply: eal_vfio_mp_sync

The secondary process detailed output:

sudo bin/test-memif-client --proc-type=auto --file-prefix=pmd2 --vdev=net_memif0,id=0,role=client,socket=/tmp/memif_test.sock,socket-abstract=no --socket-mem=512 --legacy-mem --log-level=eal,8 
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 0 on socket 0
EAL: Detected lcore 5 as core 1 on socket 0
EAL: Detected lcore 6 as core 2 on socket 0
EAL: Detected lcore 7 as core 3 on socket 0
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 8 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Auto-detected process type: SECONDARY
EAL: Checking presence of .so 'librte_eal.so.21.0'
EAL: Checking presence of .so 'librte_eal.so.21'
EAL: Checking presence of .so 'librte_eal.so'
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/pmd2/mp_socket_77819_23c2a1740b47
EAL: DPAA Bus not present. Skipping.
EAL: VFIO PCI modules not loaded
EAL: request: bus_vdev_mp
EAL: msg: bus_vdev_mp
EAL: msg: bus_vdev_mp
EAL: Bus dpaa_bus wants IOVA as 'DC'
EAL: Bus fslmc wants IOVA as 'DC'
EAL: Bus pci wants IOVA as 'DC'
EAL: Buses did not request a specific IOVA mode.
EAL: Module /sys/module/rte_kni not found! error 2 (No such file or directory)
EAL: IOMMU is not available, selecting IOVA as PA mode.
EAL: Selected IOVA mode 'PA'
EAL: Probing VFIO support...
EAL: request: eal_vfio_mp_sync
EAL: msg: eal_vfio_mp_sync
EAL: VFIO support initialized
EAL: Ask a virtual area of 0x2e000 bytes
EAL: Virtual area found at 0x100007000 (size = 0x2e000)
EAL: Setting up physically contiguous memory...
EAL: Setting maximum number of open files to 1048576
EAL: Ask a virtual area of 0x1000 bytes
EAL: Virtual area found at 0x100035000 (size = 0x1000)
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x140000000 (size = 0x800000000)
EAL: VA reserved for memseg list at 0x140000000, size 800000000
EAL: Ask a virtual area of 0x1000 bytes
EAL: Virtual area found at 0x7f8e7b87b000 (size = 0x1000)
EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in the kernel.
EAL:    This may cause issues with mapping memory into secondary processes
EAL: Analysing 2 files
EAL: Main lcore 0 is ready (tid=7f8e77d50c00;cpuset=[0])
EAL: lcore 1 is ready (tid=7f8e76d49400;cpuset=[1])
EAL: lcore 2 is ready (tid=7f8e76548400;cpuset=[2])
EAL: lcore 4 is ready (tid=7f8e75546400;cpuset=[4])
EAL: lcore 3 is ready (tid=7f8e75d47400;cpuset=[3])
EAL: lcore 5 is ready (tid=7f8e74d45400;cpuset=[5])
EAL: lcore 7 is ready (tid=7f8e5f7fe400;cpuset=[7])
EAL: lcore 6 is ready (tid=7f8e5ffff400;cpuset=[6])
EAL: PCI device 0000:00:1f.6 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 8086:15d8 net_e1000_em
EAL:   Not managed by a supported kernel driver, skipped
EAL: No legacy callbacks, legacy socket not created
pmd device init is called
Get net_memif eth device successfully port id : 0
Get eth dev info successfully
 port id : 0ethInfo : net_memif
Configure eth dev successfully
Adjust eth dev rx tx desc successfully
Setup rx queue successfully socket id : -1
Setup tx queue successfully socket id : -1
rte_eth_promiscuous_enable success 
Eth dev is started 
Port 0 Link Down

link up error Resource temporarily unavailable
Port 0 Link Down

link up error Resource temporarily unavailable
Port 0 Link Down

link up error Resource temporarily unavailable
Port 0 Link Down

The third application output (running server) :

sudo bin/test-memif-server --proc-type=auto --file-prefix=pmd1 --vdev=net_memif0,id=0,role=server,socket=/tmp/memif_test.sock,socket-abstract=no --socket-mem=512 --log-level=eal,8
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 0 on socket 0
EAL: Detected lcore 5 as core 1 on socket 0
EAL: Detected lcore 6 as core 2 on socket 0
EAL: Detected lcore 7 as core 3 on socket 0
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 8 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Auto-detected process type: PRIMARY
EAL: Checking presence of .so 'librte_eal.so.21.0'
EAL: Checking presence of .so 'librte_eal.so.21'
EAL: Checking presence of .so 'librte_eal.so'
EAL: Detected static linkage of DPDK
EAL: Ask a virtual area of 0x7000 bytes
EAL: Virtual area found at 0x100000000 (size = 0x7000)
EAL: Multi-process socket /var/run/dpdk/pmd1/mp_socket
EAL: DPAA Bus not present. Skipping.
EAL: VFIO PCI modules not loaded
EAL: Bus dpaa_bus wants IOVA as 'DC'
EAL: Bus fslmc wants IOVA as 'DC'
EAL: Bus pci wants IOVA as 'DC'
EAL: Buses did not request a specific IOVA mode.
EAL: Module /sys/module/rte_kni not found! error 2 (No such file or directory)
EAL: IOMMU is not available, selecting IOVA as PA mode.
EAL: Selected IOVA mode 'PA'
EAL: No free hugepages reported in hugepages-2048kB
EAL: No available hugepages reported in hugepages-2048kB
EAL: Probing VFIO support...
EAL:   IOMMU type 1 (Type 1) is supported
EAL:   IOMMU type 7 (sPAPR) is not supported
EAL:   IOMMU type 8 (No-IOMMU) is not supported
EAL: VFIO support initialized
EAL: Ask a virtual area of 0x2e000 bytes
EAL: Virtual area found at 0x100007000 (size = 0x2e000)
EAL: Setting up physically contiguous memory...
EAL: Setting maximum number of open files to 1048576
EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
EAL: Detected memory type: socket_id:0 hugepage_sz:2097152
EAL: Creating 2 segment lists: n_segs:32 socket_id:0 hugepage_sz:1073741824
EAL: Ask a virtual area of 0x1000 bytes
EAL: Virtual area found at 0x100035000 (size = 0x1000)
EAL: Memseg list allocated at socket 0, page size 0x100000kB
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x140000000 (size = 0x800000000)
EAL: VA reserved for memseg list at 0x140000000, size 800000000
EAL: Ask a virtual area of 0x1000 bytes
EAL: Virtual area found at 0x940000000 (size = 0x1000)
EAL: Memseg list allocated at socket 0, page size 0x100000kB
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x980000000 (size = 0x800000000)
EAL: VA reserved for memseg list at 0x980000000, size 800000000
EAL: Creating 4 segment lists: n_segs:8192 socket_id:0 hugepage_sz:2097152
EAL: Ask a virtual area of 0x61000 bytes
EAL: Virtual area found at 0x1180000000 (size = 0x61000)
EAL: Memseg list allocated at socket 0, page size 0x800kB
EAL: Ask a virtual area of 0x400000000 bytes
EAL: Virtual area found at 0x1180200000 (size = 0x400000000)
EAL: VA reserved for memseg list at 0x1180200000, size 400000000
EAL: Ask a virtual area of 0x61000 bytes
EAL: Virtual area found at 0x1580200000 (size = 0x61000)
EAL: Memseg list allocated at socket 0, page size 0x800kB
EAL: Ask a virtual area of 0x400000000 bytes
EAL: Virtual area found at 0x1580400000 (size = 0x400000000)
EAL: VA reserved for memseg list at 0x1580400000, size 400000000
EAL: Ask a virtual area of 0x61000 bytes
EAL: Virtual area found at 0x1980400000 (size = 0x61000)
EAL: Memseg list allocated at socket 0, page size 0x800kB
EAL: Ask a virtual area of 0x400000000 bytes
EAL: Virtual area found at 0x1980600000 (size = 0x400000000)
EAL: VA reserved for memseg list at 0x1980600000, size 400000000
EAL: Ask a virtual area of 0x61000 bytes
EAL: Virtual area found at 0x1d80600000 (size = 0x61000)
EAL: Memseg list allocated at socket 0, page size 0x800kB
EAL: Ask a virtual area of 0x400000000 bytes
EAL: Virtual area found at 0x1d80800000 (size = 0x400000000)
EAL: VA reserved for memseg list at 0x1d80800000, size 400000000
EAL: Allocating 1 pages of size 1024M on socket 0
EAL: Trying to obtain current memory policy.
EAL: Setting policy MPOL_PREFERRED for socket 0
EAL: Restoring previous memory policy: 0
EAL: Added 1024M to heap on socket 0
EAL: TSC frequency is ~2000000 KHz
EAL: Main lcore 0 is ready (tid=7fb885decc00;cpuset=[0])
EAL: lcore 1 is ready (tid=7fb884de5400;cpuset=[1])
EAL: lcore 3 is ready (tid=7fb87f7fe400;cpuset=[3])
EAL: lcore 2 is ready (tid=7fb87ffff400;cpuset=[2])
EAL: lcore 5 is ready (tid=7fb87e7fc400;cpuset=[5])
EAL: lcore 4 is ready (tid=7fb87effd400;cpuset=[4])
EAL: lcore 7 is ready (tid=7fb87d7fa400;cpuset=[7])
EAL: lcore 6 is ready (tid=7fb87dffb400;cpuset=[6])
EAL: PCI device 0000:00:1f.6 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 8086:15d8 net_e1000_em
EAL:   Not managed by a supported kernel driver, skipped
EAL: No legacy callbacks, legacy socket not created
8191
pmd device init is called
Get net_memif eth device successfully port id : 0
Get eth dev info successfully
 port id : 0ethInfo : net_memif
Configure eth dev successfully
Adjust eth dev rx tx desc successfully
Setup rx queue successfully socket id : -1
Setup tx queue successfully socket id : -1
rte_eth_promiscuous_enable success 
Eth dev is started 
Port 0 Link Down

Port 0 Link Down

Port 0 Link Down

Port 0 Link Up - speed 10000 Mbps - full-duplex

Application is started 
Server receiving
Mustafa
  • 147
  • 1
  • 12
  • can you please highlight the DPDK documentation for `According to the Dpdk pmd multiprocess application specification, I have to initialize memif vdev at primary process and use the device at secondary process`. memif is normally used between 2 DPDK primary process or DPDK & VPP. With respect to your commands the second application is using `--proc-type=auto --file-prefix=pmd1` making it DPDk primary process. Now with respect to the error please share `DPDK version, binary format, platform used`. memif works for me between testpmd, l2fwd, pktgen, vpp and my custom application. – Vipin Varghese Sep 22 '22 at 04:46
  • @VipinVarghese According to [this](https://doc.dpdk.org/guides/prog_guide/multi_proc_support.html) link. You can use memif in secondary process. This is mandatory implementation in my case. Also maybe you missed the point, I am running these two processes at the same time. So this argument `--proc-type=auto --file-prefix=pmd1`, you can run the process as secondary process. I actually not facing with error, but I am seeing that the secondary process (client) memif cannot link up the eth device. rte_eth_link_get returns with ` Resource temporarly unavalable `. – Mustafa Sep 22 '22 at 07:27
  • @VipinVarghese also So server memif (which is third independend application, itself is a primary process ) cannot get anything over memif. – Mustafa Sep 22 '22 at 07:28
  • checking the link shared there is no mention of memif in `43. Multi-process Support`. `I am not denying or accepting one should or can run memif in secondary, since memif internally makes certain offset calculation which might fail for secondary especially with ASLR not enabled`. With respect to DPDK primary-secondary model `devices are shared and huge pages are shared` . hence ideal command lien for secondary should be `priamry device args + new secondary device args`. This will conflict. So I am open for a live debug, hope it makes things clear for you now. – Vipin Varghese Sep 22 '22 at 09:41
  • @VipinVarghese, I understand. I also added the detailed output. Could you please look at the outputs. If you want I can share the code base with you. Can you see any problem at outputs ? – Mustafa Sep 22 '22 at 12:19
  • I got some time to check the memif driver code, as I had pointed out in my earlier comments, the way it is written for primary-secondary and your expectation is different. I can spend some time to help you understand the code and right flow. Please setup a live debug in zoom, google meet, or skype – Vipin Varghese Sep 22 '22 at 12:30
  • I waited for your meeting to explain, since I did not receive any I have updated the answer with an explanation. Please read and cross check if answers your queries. If you find it helpful please accept and upvote to help others – Vipin Varghese Sep 23 '22 at 05:17

1 Answers1

1

There are some gaps in understanding DPDK Primary-Secondary process and how memif PMD works in DPDK. Let me try to explain

  • I implementing the Poll Mode Driver(PMD) memif(vdev=net_memif) virtual device in application.

[Answer] based on the code snippet, memif PMD is getting used in custom applications. There is no code snippet indicating memif PMD is modified or enhanced for the requirement.

  • I have to run memif as a client in a secondary process

[Answer] yes, memif can be used in the secondary process. That is, to a primary process that creates a memif interface as server or client; a secondary process can be attached. From the secondary process one can allow rx_burst and tx_burst too. But the memif device instance is created by the primary to be accessed by secondary.

Can one create a new memif interface in secondary? Answer is NO, because of the memif PMD probe code always checks process type as secondary and tries to attach to memif interface in secondary.

File: drivers/net/memif/rte_eth_memif.c

Function: rte_pmd_memif_probe

Code:

    if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
        eth_dev = rte_eth_dev_attach_secondary(name);
        if (!eth_dev) {
            MIF_LOG(ERR, "Failed to probe %s", name);
            return -1;
        }

        eth_dev->dev_ops = &ops;
        eth_dev->device = &vdev->device;
        eth_dev->rx_pkt_burst = eth_memif_rx;
        eth_dev->tx_pkt_burst = eth_memif_tx;

        if (!rte_eal_primary_proc_alive(NULL)) {
            MIF_LOG(ERR, "Primary process is missing");
            return -1;
        }

        eth_dev->process_private = (struct pmd_process_private *)
            rte_zmalloc(name,
                sizeof(struct pmd_process_private),
                RTE_CACHE_LINE_SIZE);
        if (eth_dev->process_private == NULL) {
            MIF_LOG(ERR,
                "Failed to alloc memory for process private");
            return -1;
        }

        rte_eth_dev_probing_finish(eth_dev);

        return 0;
    }

hence as shared in the comment, `one can not create a new memif interface via secondary DPDK process but use the already existing device from primary.

  • According to the Dpdk pmd multiprocess application specification, I have to initialize memif vdev at the primary process and use the device at the secondary process.

[Answer] as pointed out in the comments DPDK multi Process link does not talk about memif initialization at all.

  • With this information, I can implement the server memif vdev which can run at a secondary process.

[Answer] as explained above this is not possible with the current memif PMD code.

**- Do I have to initialize, configure and run memif client eth device on the primary process to be able to run memif client on the secondary process?

[Answer] Yes, using DPDK primary application one has to initialize and configure the memif just like regular DPDK ports.**

  • If the client memif eth device can be run as same as the server, what am I missing?

[Answer] once the primary dpdk application configures and starts the memif, the secondary application can make use of rx_burst and tx_burst.

Recommendation: As shared in the comment, the ideal way of using memif is between 2 DPDK processes in the following combination

  1. primary process A (server) to primary process B (client)
  2. primary process A (server) to secondary process B (client)
  3. Secondary process A (server) to primary process B (client)
  4. secondary process A (server) to secondary process B (client)
Vipin Varghese
  • 4,540
  • 2
  • 9
  • 25
  • Hi, I am very sorry I could not set a meeting for live debug. I appreciate your help. There is a one thing that I am seeing it as an exception. I am sharing a link of my basic test codes. Also I added commands for the test. If you can run the code as I do, you can understand. [memif-examles-with-command](https://fromsmash.com/memif). – Mustafa Sep 23 '22 at 09:21
  • For test-memif-dummy(Primary process) : **sudo ./test-memif-dummy --proc-type=primary --file-prefix=pmd2 --vdev=net_memif0,id=0,role=client,socket=/tmp/memif_test.sock,socket-abstract=no --log-level=eal,8** For test-memif-client(Secondary process) : **sudo ./test-memif-client --proc-type=auto --file-prefix=pmd2** For test-memif-server(secondary process) : **sudo ./test-memif-server --proc-type=auto --file-prefix=pmd1 --vdev=net_memif0,id=0,role=server,socket=/tmp/memif_test.sock,socket-abstract=no --log-level=eal,8** – Mustafa Sep 23 '22 at 10:01
  • @Mustafa I did not follow you, can you explain. it will be much easier for me if you come for debug – Vipin Varghese Sep 23 '22 at 10:17
  • you can share the google meet or team invite here I will join – Vipin Varghese Sep 23 '22 at 13:27
  • Thank you very much again. It was very helpful. Thank you for sparing your time and sharing your knowledge. – Mustafa Sep 23 '22 at 14:54