10

I've implemented a dense optical flow algorithm and I want to visualize it with following color model

enter image description here

(color denotes direction of flow at some point, intensity denotes length of displacement vector)

I've implemented a dirty version of the visualization 

def visualizeFlow(u, v):
    colorModel = cv2.imread('../colormodel.png')
    colorModelCenter = (colorModel.shape[0]/2, colorModel.shape[1]/2)
    res = np.zeros((u.shape[0], u.shape[1], 3), dtype=np.uint8)
    mag = np.max(np.sqrt(u**2 + v**2)) 
    if mag == 0:
        return res, colorModel
    for i in xrange(res.shape[0]):
        for j in xrange(res.shape[1]):
            res[i, j] = colorModel[
                        colorModelCenter[0] + (v[i, j]/mag*colorModelCenter[0]),
                        colorModelCenter[1] + (u[i, j]/mag*colorModelCenter[1])
                    ]
    return res, colorModel

It produce nice in general case pictures but it really slow

enter image description here

So my question is can anyone help me make this visualization faster? If somebody knows a better way to visualize dense flow it may be cool

GPPK
  • 6,546
  • 4
  • 32
  • 57
Daiver
  • 1,488
  • 3
  • 18
  • 47
  • what does "really slow" mean? – Micka Mar 06 '15 at 12:30
  • @Micka visualization of flow with size 2637 x 1300 takes a few minutes. It very important to me because i want to use it inside warping iteration to trace how flow changes inside optimization process. And at least level of pyramid visualization may take to much time. – Daiver Mar 06 '15 at 12:44
  • If you sum the height and width, that's how many times your for loop is running. Can you see the reason its slow? – GPPK Mar 06 '15 at 12:54
  • @GPPK Just now i remove /mag*colorModelCenter from cycle (and replace it by single multiplication) and it seems that function becomes faster. I suppose that another problem is multiple call of matrix items in python code. – Daiver Mar 06 '15 at 13:06
  • 1
    if you call this often you probably shouldnt imread the model in each call but load once and pass. but im not in python so not sure. – Micka Mar 06 '15 at 13:10
  • @Micka it good advice but nested cycles takes much more time than image reading – Daiver Mar 06 '15 at 13:12
  • sure, wasnt meant as a solution but a hint for additional important speedup. – Micka Mar 06 '15 at 13:15
  • This looks very cool. Do you have any videos showing a video feed vs the colourised optical flow ? I'm trying to find a good comparison video. – rollsch Jan 18 '18 at 02:03
  • @rolls unfortunately no =( But i'm pretty sure that it's easy to do – Daiver Jan 18 '18 at 11:33

3 Answers3

16

Code from OpenCV's tutorial:

import cv2
import numpy as np

# Use Hue, Saturation, Value colour model 
hsv = np.zeros(im1.shape, dtype=np.uint8)
hsv[..., 1] = 255

mag, ang = cv2.cartToPolar(flow[..., 0], flow[..., 1])
hsv[..., 0] = ang * 180 / np.pi / 2
hsv[..., 2] = cv2.normalize(mag, None, 0, 255, cv2.NORM_MINMAX)
bgr = cv2.cvtColor(hsv, cv2.COLOR_HSV2BGR)
cv2.imshow("colored flow", bgr)
cv2.waitKey(0)
cv2.destroyAllWindows()

enter image description here

gerrit
  • 24,025
  • 17
  • 97
  • 170
Mastergalen
  • 4,289
  • 3
  • 31
  • 35
  • what is the dimension of flow ? I have passed my computed u and v as arguments (dim 480 x 640). As a result it has thrown up an error since hsv[...,0].shape =480 and the assignment is not right. – creative Sep 14 '19 at 09:59
5

You might wanna check the awesome flow_vis package. Quoting from their page:

  1. pip install flow_vis

  2. Then in your code:

import flow_vis
flow_color = flow_vis.flow_to_color(flow_uv, convert_to_bgr=False)
nivniv
  • 3,421
  • 5
  • 33
  • 40
3

If you use function provided by opencv your code will run faster. The optical flow visualization works as follow:

  • Transform u and v motion components matrices into polar coordinate. Applying cartToPolar function (x array = u array, y array = v array) will get you angle and magnitude matrices of your motion vectors.

The final colour visualization can than be found by an inverse HSV to RGB transformation, where the angle matrice corresponde to the Hue (H) channel and the magnitude to the saturation (S) the value (V) is set to maxima. ( In your example the value and saturation channels are swapped).

  • Merge the magnitude, angle and a matrice filled with 1 to a CV_32FC3 channel matric using merge or mixChannels.

  • Apply cvtColor with the flag CV_HSV2BGR. Note angle matric is in degrees and magnitude has to be rescaled to fit i [0,1] which can be done by dividing it by the maximun of the magnitude using e.g. MinMaxLoc

Tobias Senst
  • 2,665
  • 17
  • 38