We are trying to synchronize 2 different streams with respect to their RTP timestamps. More specifically, we are sending two identical uncompressed video streams on port 5004 and 5005, at 24 fps with a clock-rate of 90000Hz, except that there is an offset of 90000/8=11250 in the RTP timestamps of the second stream. Therefore, when displaying the two streams with below GStreamer pipeline, we would like the images of the second stream to appear 1/8 second (=3 frames) later.
caps="application/x-rtp,media=(string)video,clock-rate=90000,encoding-name=(string)RAW,sampling=(string)YCbCr-4:2:2,depth=(string)8,width=(string)640,height=(string)480,payload=96"; \
GST_DEBUG="*jitter*:4" \
gst-launch-1.0 udpsrc buffer-size=16000000 port=5004 caps="${caps}" ! rtpjitterbuffer mode=0 ! rtpvrawdepay ! queue ! videoconvert ! fpsdisplaysink \
udpsrc buffer-size=16000000 port=5005 caps="${caps}" ! rtpjitterbuffer mode=0 ! rtpvrawdepay ! queue ! videoconvert ! fpsdisplaysink
Unfortunately, it is not the case: rtpjitterbuffer seem to compute the PTS and DTS timestamps of the packets that it receives from the RTP timestamps, but it uses a different "base RTP time" for both streams:
0:00:00.021740027 17400 0x556306a95120 INFO rtpjitterbuffer rtpjitterbuffer.c:809:rtp_jitter_buffer_calculate_pts: resync to time 0:00:00.000000000, rtptime 0:00:00.125000000
0:00:00.021752755 17400 0x556306a95180 INFO rtpjitterbuffer rtpjitterbuffer.c:809:rtp_jitter_buffer_calculate_pts: resync to time 0:00:00.000000000, rtptime 0:00:00.000000000
From the RTP Readme, the formula to compute the RTP time from the running time looks like this:
RTP = ((RT - RT-base) * clock-rate / GST_SECOND) + RTP-offset
So the formula that is used by the rtpjitterbuffer at the receiver-end probably looks like this:
RT - RT-base = (RTP - RTP-offset) * GST_SECOND / clock-rate
How can I make so that the rtpjitterbuffer computes the timestamps (RT) with the same value of RT-base and RTP-offset for both streams ?
NOTE: we do not want to use RTCP or any other synchronization mechanism: the RTP timestamps are already based on the same clock (we work on localhost) with an offset of 0.
EDIT: Could the rfc7273-sync property of the rtpjitterbuffer with a-mediaclk=”direct=0” in the caps help ?
