Combining multiple perspectiveTransforms into one transform with opencv

Question

I have an application that has two perspective transforms obtained from two findHomography calls that get applied in succession to a set of points (python):

pts = np.float32([ [758,141],[769,141],[769,146],[758,146] ]).reshape(-1,1,2)
pts2 = cv2.perspectiveTransform(pts, trackingM)
dst = cv2.perspectiveTransform(pts2, updateM)

I would like to combine this into a single transformation. I've tried the following but the transformation is not correct:

M = trackingM * updateM
dst = cv2.perspectiveTransform(pts, M)

How can I combine two matrix transforms into a single transform? For now I'm prototyping in python. A C++ solution in addition to python would be a bonus.

score 1 · Answer 1 · answered Jan 26 '18 at 01:47

1

In numpy, np.multiply(M, N) is elementwise-product, while np.dot(M,N) is dot-product. I think, in your case, you should choose np.dot.

For example:

>>> import numpy as np 
>>> x = np.array([[1,2],[3,4]])

## elementwise-product 
>>> x*x
array([[ 1,  4],
       [ 9, 16]])
>>> np.multiply(x,x)
array([[ 1,  4],
       [ 9, 16]])

## dot-product
>>> x.dot(x)
array([[ 7, 10],
       [15, 22]])
>>> np.dot(x,x)
array([[ 7, 10],
       [15, 22]])

answered Jan 26 '18 at 01:47

Kinght 金

17,681
4
60
74

It is not obvious why the dot-product would combine the two transforms into one. Can you elaborate? – Demosthenes Jan 26 '18 at 07:51
@Demosthenes If you have learned advanced math, then you should know how to calc `dot_product`. Or go to wiki to learn: https://en.wikipedia.org/wiki/Dot_product – Kinght 金 Jan 26 '18 at 07:57
I'm slightly offended by that, having a PhD in computer science, but you can't know that. I didn't say the answer was wrong (I didn't down-vote it), I just suggested it might be worthwhile explaining why this combines the two transformations, since that was the original question. – Demosthenes Jan 26 '18 at 08:28
If I said `obviously, you should use dot-product other than element-wise-product`, it also doesn't help. I can't explain why, just intuition, so I give my solution. Or, let me ask first, why use `element-wise product`? – Kinght 金 Jan 26 '18 at 08:44
I tried the dot product operation and can verify the resulting transform was ill formed. I did find a solution though which I've posted. – Glenn Jan 26 '18 at 18:01

Sunil · Answer 2 · 2022-11-08T11:55:50.530

Let us say, we have a series of Perspective Transformation as follows. Let tⁱ_j be the perspective transform matrix from image_i to image_j

image_1 -- t¹₂ --> image_2 -- t²₃ --> .... -- t^N-1_N --> image_N

The point p₁ in image_1 would be transformed to point p₂ in image_2 as p₂ = t¹₂.p₁

The point p₂ in image_2 would be transformed to point p₃ in image_3 as p₃ = t²₃.p₂ = t²₃.t¹₂.p₁

Hence p_N = t^N-1_N...t²₃.t¹₂.p₁

Hence the resultant transformation matrix t¹_N = t^N-1_N...t²₃.t¹₂

Hence you can get M = np.dot(updateM, trackingM)

Note that all products are matrix (np.dot) products.

score 0 · Answer 3 · answered Jan 26 '18 at 18:00

I found a clue to the answer in this post. In essence my original code was doing element-wise multiplication as pointed out by @Silencer. The trick is to convert the transform to a matrix before doing the multiply:

M = np.matrix(updateM) * np.matrix(trackingM)
dst = cv2.perspectiveTransform(pts, M)

The order of the operands is significant. One can think of the above as applying the updateM transformation to the trackingM transform to match the order stated in the original problem.

Combining multiple perspectiveTransforms into one transform with opencv

3 Answers3