0

I need to find vectors that best describe the difference between two images, for example two consecutive frames of an animation. At the moment I have code that works, but that also takes awfully long to complete. My hope is that one or the other knows a trick to speed it up considerably.

What I have so far is this:

    nsh = 20
    nqu = 45
    nn = np.square(2 * nqu + 1)

    kernel = np.ones((nqu, nqu))
    sy = convolve2d(image1, kernel, 'same')
    sy2 = convolve2d(np.square(image1), kernel, 'same')
    sigy = sy2 - np.square(sy)/nn
    sho = np.zeros_like(image1)

    kernel = np.ones((nsh, nsh))
    for ish in range(-nsh, nsh+1):
        for jsh in range(-nsh, nsh+1):
            tmp = np.roll(np.roll(image2, ish, axis=0), jsh, axis=1)
            sx = convolve2d(tmp, kernel, 'same')
            sx2 = convolve2d(np.square(tmp), kernel, 'same')
            sxy = convolve2d(tmp, kernel, 'same') * image1
            sigx = sx2 - np.square(sx)/nn

            corq = np.square(sxy - sx * sy / nn) / (sigx * sigy)
            boole = corq > corqx
            if not boole.any():
                continue
            sho[boole, :] = [ish, jsh]
            # dpo[boole] = sy[boole] - sx[boole]

image1 and image2 are the two consecutive frames and 2d arrays. With the scipy function convolve2d I want to accumulate the values over a certain area to reduce noise that would potentially deteriorate my result.

As you might guess the part that's taking ages is the double loop where I shift my other image around in order to find the position with the best match.

Is there a way to get rid of the two loops?

Martin K
  • 33
  • 4

1 Answers1

1

It might very well be that I don't get why you are using all the convolutions, therefore I also don't have solution for getting rid of the double loop but I have 2 thoughts about your task.

  1. From your description you would like to get an overall shift from one image to the next one, correct? If that is correct, maybe a 2d correlation will give you a performance increase https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.correlate2d.html#scipy.signal.correlate2d.

  2. Since you anyway try to get rid of noise you are most likely not interested in very small details of the images. Therefore another idea would be to generate scaled down images and perform your evaluations on these.

Flow
  • 551
  • 1
  • 3
  • 9
  • Thanks for your suggestions. You were right, a few of my convolutions could be saved by using correlate2d. Together with thinning my input fields the code got considerably faster. Unfortunately, the speed gain was too small for my application. I'll probably resort to a C type code and numba. – Martin K Sep 06 '22 at 13:35