How to synchronize two USB cameras to use them as Stereo Camera?

Question

I am trying to implement object detection using Stereo Vision in OPENCV. I am using two Logitech-C310 camera for this. But I am not getting synchronized frames with two cameras. Time difference between two cameras frame capture is also not same.

• How synchronization can be done ?

• In Stereo Cameras like Bumblebee, Minoru etc. do we need to synchronize ?

---

Thanks for your response.

I am trying to implement person tracking with a moving robotic platform. I am using cvQueryFrame(capture) to capture each frame from both cameras one by one in a loop. Here is the part of code that I am using:

CvCapture* capture_1 = cvCreateCameraCapture(0);
Cvcapture* capture_2 = cvCreateCameraCapture(1);
for(i=1;i<=20;i++)
{
 frame_1= cvQueryFrame(capture_1);
 frame_2= cvQueryFrame(Capture_2);

//processing of frames//

}

even if someone moves with moderate speed in front of camera the difference between frame_1 and frame_2 is visible.

Is this delay because of cvQueryFrame(capture)?

Answer 1

TL;DR

See my last code snippet "A simple workaround". That's how I did it.

---

Although I didn't work with CvCapture but with VideoCapture and not C++ but Python, my solution might still apply to your problem. I also wanted to capture synchronized stereo images with OpenCV.

A naive attempt might be:

vidStreamL = cv2.VideoCapture(0)
vidStreamR = cv2.VideoCapture(2)

_, imgL = vidStreamL.read()
_, imgR = vidStreamR.read()

vidStreamL.release()
vidStreamR.release()

Problem 1: The second camera is only triggered after the first image is captured and retrieved from the camera, which takes some time.

A better way is to grab the frame first (tell the cameras to pin down the current frame) and to retrieve it afterwards:

vidStreamL = cv2.VideoCapture(0)
vidStreamR = cv2.VideoCapture(2)

vidStreamL.grab()
vidStreamR.grab()
_, imgL = vidStreamL.retrieve()
_, imgR = vidStreamR.retrieve()

vidStreamL.release()
vidStreamR.release()

Problem 2: I still measured differences of about 200 ms (filming a watch with milliseconds). The reason is an internal capture buffer, described here. Unfortunately, it cannot always be easily deactivated (at least in my OpenCV version).

A simple workaround is to grab frames multiple times until the capture buffer is empty before retrieving the actual images:

vidStreamL = cv2.VideoCapture(0)
vidStreamR = cv2.VideoCapture(2)

for i in range(10):
    vidStreamL.grab()
    vidStreamR.grab()
_, imgL = vidStreamL.retrieve()
_, imgR = vidStreamR.retrieve()

vidStreamL.release()
vidStreamR.release()

This solution works well for my case. I couldn't see any measurable difference (< 10 ms).

(Problem 3:) Technically, the cameras are still not synchronized. Normal USB webcams won't be able to do that. But more professional cameras often have an external trigger to actually control when to start capturing a frame. This is beyond the scope of this post.

Answer 2

This is not a coding solution and requires hardware changes.

Trying to take one camera as the reference and read the other camera into memory and read it out with sufficient delay to be synchronous with the first, happened to be a lot of trouble and crashes when there is too much movement.

As a result, I bought an HDMI switch with PIP and selected the left and right frame to be side by side. This produces a live stereo image and doesn't crash every 30 seconds. The single output stream can be manipulated in open CV to resize it or stretch or crop it.

Stage 2 of this hardware-based plan is to try and find something that will do the job but be physically smaller. I have found this in the analogue domain with a drone 3d camera kit. Drones use analogue to avoid latency as that could result in a crash if the operator is flying first person view using goggles.

Sorry for the inconvenience as it's a hardware solution but I am really fed up of trying to get a Raspberry Pi to do something that should be easy but isn't. Posted it in case it helps some other person to not waste too much time and is capable of looking for alternate solutions.

Answer 3

As far as I can remember, the two cameras form the Minoru are not synchronized. The exact communication pattern that you need to implement will depend on the driver that is used to access the cameras, so will be platform dependant and you need to provide more information to get accurate answers.

Now, another question is: do you really need synchronization? I understand that sterovision is obviously simpler when the cams are perfectly in sync, but at something like 30 fps (or even 15), an object needs to move really fast before some distortion occur.