My DPDK application must fragment IPv4 packets before appending a trailer and transmitting the fragments with their trailers. Ideally it should do this with zero copies of the packet body.
For each fragment generated during fragmentation, the DPDK IPv4 fragmentation function creates:
- One direct mbuf containing a new IPv4 header.
- One indirect mbuf that references the original (pre-fragmentation) packet data.
The indirect mbuf is chained to the direct mbuf. As I can't change the original data to insert the trailer, I think I should insert a third segment at the end of the chain, after the indirect mbuf, to contain the trailer data (and that it's OK to do this in DPDK).
This results in an assertion failure as described below.
I'm using an NXP 2160 SoC (DPAA2 PMD and associated drivers). I've tried allocating and chaining the trailer data as follows (N.B. return value checks are omitted for brevity - they return OK)
struct rte_mbuf *fragment = fragments[i];
struct rte_mbuf *trailer = rte_pktmbuf_alloc(mbufpool_direct);
void *d = rte_pktmbuf_append(trailer, sizeof(some_trailer_data));
rte_memcpy(d, some_trailer_data, sizeof(some_trailer_data));
rte_pktmbuf_chain(pkt, trailer);
This seems to create the mbuf chain I think I need:
0 mbuf 0x17f47e700 next 0x17e512800 segs 3 data_len 20 ind=N (fragment header)
1 mbuf 0x17e512800 next 0x17f47af80 segs 1 data_len 1176 ind=Y (indirect packet payload)
2 mbuf 0x17f47af80 next (nil) segs 1 data_len 10 ind=N (trailer)
However, the code panics in the __rte_mbuf_raw_sanity_check function during a subsequent (much later) call to rte_pktmbuf_alloc. This panic only occurs when I append the trailer as shown above. The assertion and backtrace seems to indicate that a mbuf retrieved from the mempool for allocation has the next field set:
PANIC in __rte_mbuf_raw_sanity_check():
line 569 assert "m->next == ((void *)0)" failed
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x0000fffff7a9baac in __GI_abort () at abort.c:79
#2 0x0000fffff7cb2be4 in __rte_panic () from /usr/local/lib/aarch64-linux-gnu/librte_eal.so.23
#3 0x0000aaaaaaaac804 in __rte_mbuf_raw_sanity_check (m=0x17f47e700) at /usr/local/include/rte_mbuf.h:569
#4 rte_mbuf_raw_alloc (mp=0x17f817800) at /usr/local/include/rte_mbuf.h:602
#5 0x0000aaaaaaaac9bc in rte_pktmbuf_alloc (mp=0x17f817800) at /usr/local/include/rte_mbuf.h:908
#6 0x0000aaaaaaab06ac in add_trailer (csa_idx=1234, pkt=0x17f478140, cxt=0xaaaaaaae4440 <appcxt+128>)
I'm trying to understand whether my method for adding a trailer is incorrect / unsupported or whether it may be an issue with the DPAA2 PMD.
