We're trying to capture footage from multiple live cameras. We want to end up with a video for each camera where frame N from each video corresponds to the same moment in time (+/- a few milliseconds). From our understanding, multiqueue should do this for us. However, the following pipeline doesn't do what we want:
multiqueue max-size-buffers=1 sync-by-running-time=true name=mqueue \
tcambin serial=33810032 name=source \
! timecodestamper post-messages=true \
! timeoverlay shading-value=255 shaded-background=true font-desc=\"Sans, 10\" time-mode=0 \
! video/x-raw,format=BGRx,width=1920,height=1080,framerate=30/1 \
! mqueue.sink_1 \
tcambin serial=35910547 name=source1 \
! timecodestamper post-messages=true \
! timeoverlay shading-value=255 shaded-background=true font-desc=\"Sans, 10\" time-mode=0 \
! video/x-raw,format=GRAY8,width=1920,height=1080,framerate=15/1 \
! mqueue.sink_2 \
mqueue.src_1 \
! videoconvert \
! x264enc speed-preset=1 pass=5 bitrate=100000 threads=6 \
! mp4mux \
! filesink location=out1.mp4 \
mqueue.src_2 \
! videoconvert \
! x264enc speed-preset=1 pass=5 bitrate=100000 threads=6 \
! mp4mux \
! filesink location=out2.mp4
Note that one camera is set for 30fps, the other for 15. Because we have a queue depth of 1 on the multiqueue, I would expect that the downstream encoding parts of the pipeline (src_1 & src_2) should be pulling off one frame each, which would unblock the upstream sinks, which would then get whatever image is current on the camera.
Instead, we're getting videos where src_1 is has twice as many frames as src_2. Our assumption is that multiqueue should prevent this behavior.
