TL;DR: Check the size of updsrc buffer if you insist on using gstreamer. I recommend using a Ravenna implementation instead.
The question has a lot of focus on gstreamer, which might not be needed to achieve your goal.
This answer is more about Audio over IP (AoIP) on ARM Linux in general, as are question 2. and 3.
First, you mention Video and Audio in your setup. If done professionally this would require an AV over IP solution that keeps audio and video in sync. Examples are DanteAV or something SMPTE 2110 compatible. Your approach seems fine to me for hobby projects. Keep in mind, Dante in AES67 mode has some constraints.
2. Low Latency AoIP:
Yes, it is possible to get Dante/AES67/RTP latency below 50 ms definitely, if your device has comparable specs to an NXP i.MX8.
As a reference: roundtrip latency of 4-5ms is achieved easily with Dante and professional hardware.
If your goal is: "Getting low-latency AES67 stream to RPi", I suggest to look at a Ravenna implementation for Linux. Ravenna is similar to Dante, but more AES67 compliant. Ravenna and AES67 use RTP exactly the same way.
There are two main open source Linux implementations:
- Merging Technologies provides the "original" with commercial only support. The control tool is proprietary and x86 only.
- There is a fork that contains useful patches and open source control software.
- In both cases, you still need to cross compile the code for ARM/RPi.
- Personally, I recommend using the fork.
Dante provides a Linux implementation for manufacturers but you probably won't get your hands on that: https://www.audinate.com/products/manufacturer-products/dante-embedded-platform.
3. Latency Sources
For AoIP there are different sources of latency:
- Packet time: time to account for samples in packet, ca. 1ms.
- Network: depends on your hardware and settings, ca. < 1ms to many seconds.
- Receiver Buffer: configurable, ca. < 1ms to many ms.
- Processing and DAC at the receiver.
Packet time should be 1ms, because Dante uses AES67 mandatory profile (48 kHz, 48 samples/packet). I assume your sender handles this correctly, but I can't tell.
On the network, you should use Gigabit switches and Cat 5e Cables at least.
Make sure to follow recommendations for switches and configuration (Esp. disable EEE).
If the clocks of Sender and receiver are synchronized, run tcpdump or Wireshark on both to get a good estimate of your network latency.
Filtering for payload is the easiest by port, e.g. RTP: port 5004 or port 9875 Dante: portrange 14336-14591. Unmuting/Muting is easy to spot in the capture (latency = T_send - T_recv).
If above are okay, look at receive buffer or processing at the receiver.
RPi low latency audio seems possible.
Still, you could measure the speed of ADC/DAC on the RPi as well as optimize your Linux (Ranges from going headless to real time patched kernel to building an optimized distro with Yocto in the extreme.
1., 4. Gstreamer
As said, if you are able to do so use the Ravenna Linux implementation. Its whole purpose is to efficiently receive AES67 streams. Still, some ideas about your pipeline:
The docs say udpsrc adds a configurable buffer of ca. 50 kB to max. 100 kB.
An AES67 RTP packet at 1ms packet time and 48 kHz sampling frequency has 342 bytes.
So the buffer is equivalent to 50 kB / 342 Byte * 1ms = 146ms
-> This is a likely culprit.
I think, you do not need rtpjitterbuffer. Its docs talk about "retransmission". AES67 does no retransmission of payload (AFAIK).
Also, the pipeline adds another buffer of 200 packets, that is 200ms at 1ms packet time.
Ordering etc. should still work, as AES67 uses standard RTP timestamps. But it will cost you latency and is probably not necessary on LAN.
It seems, rtpjitterbuffer is more geared towards wide area network RTP than local network AES67.
Also found this article, which uses the same pipeline as you do.
If this doesn't fix your problem, find out where exactly the latency is introduced. Also tell us how the latency was measured.
