OpenCV time to contact with optical flow

Question

I'm trying to write a program with C++ and OpenCV that calculates the remaining time (frames) until a collision with an object. As a sample video I have a camera moving towards a blackboard.

My approach on this is the following:

1. detect features to track (tried goodFeaturesToTrack() or setting points "by hand")
2. calculate optical flow via calcOpticalFlowPyrLK()
3. calculate fundamental matrix of the previous and current found features via findFundamentalMat()
4. check if fundamental matrix is correct
5. calculate epipolar lines and epipole - the focus of expansion in the video

Finally, I'd planned to use the time to contact method to calculate the remaining frames till collision.

Up until now, my biggest problem is to find the correct fundamental matrix and thus the epipole. The calculated matrix seems to be wrong as are the epipolar lines. Would my planned approach be correct? Does anyone have a working example of getting a FoE from a video or any kind of walkthrough on what I have to do?

I'd be very happy about any help at all!

Thank you!

Answer 1

I think LK optical flow will detect 2D flow only. While in principle you can deduce information from motion in a part of the image (i.e., first quadrant), fundamental matrix needs a stereo pair of images, not consecutive ones.

From the top of my head, I don't know the perfect steps to solve your problem. But I'm thinking researching structure from motion might help you, because this technique solves the problem to create 3-dimensional data from motion but without a stereo camera pair.