Convection of an image using optical flow

Question

I have two images (frame1 and frame2) and I am able to calculate u,v using opencv:

flow = cv2.calcOpticalFlowFarneback(prvs,next, 0.5, 1, 3, 15, 3, 5, 1, 0)

I want to translate frame1 using this u,v to quantify the quality of the difference using various optical flow methods. I intend to extrapolate using these u,v.

Is there a simple way to achieve this?

Oh, do you mean each point has a `(u,v)` vector attached to it? — Mateen Ulhaq, Jun 14 '17 at 03:55
yes. I guess this could solve my issue but I am not able to convert the equivalent of Point in python. https://stackoverflow.com/questions/17459584/opencv-warping-image-based-on-calcopticalflowfarneback — matsu, Jun 14 '17 at 05:24

score 8 · Accepted Answer · answered Jun 14 '17 at 07:52

One way to compute a simple translation is to average the flow:

avg_u = np.mean(flow[:, :, 0])
avg_v = np.mean(flow[:, :, 1])

This gives the vector (avg_u, avg_v) needed to translate.

Regarding your comments, it seems you want to move every pixel (x,y) to the location specified by its flow vector (u(x,y), v(x,y)).

First, generate a Cartesian grid:

height, width = flow.shape[0, 1]
R2 = np.dstack(np.meshgrid(np.arange(width), np.arange(height)))

Then, the desired mapping is simply the addition of this grid with the flow:

pixel_map = R2 + flow

Finally, perform a cv2.remap:

new_frame = cv2.remap(prev_frame, pixel_map)

Interestingly, using R2+ flow is giving worse results than R2-flow with large margin. I am using it for extrapolation. — matsu, Sep 15 '18 at 06:59

Convection of an image using optical flow

1 Answers1

Linked