Converting OpenCV's findHomography perspective matrix to iOS' CATransform3D

Question

I'd like to take the perspective transform matrix returned from OpenCV's findHomography function and convert it (either in C++ or Objective-C) to iOS' CATransform3D. I'd like them to be as close as possible in terms of accurately reproducing the "warp" effect on the Core Graphics side. Example code would really be appreciated!

From iOS' CATransform3D.h:

/* Homogeneous three-dimensional transforms. */

struct CATransform3D
{
    CGFloat m11, m12, m13, m14;
    CGFloat m21, m22, m23, m24;
    CGFloat m31, m32, m33, m34;
    CGFloat m41, m42, m43, m44;
};

Similar questions:

Apply homography matrix using Core Graphics

convert an opencv affine matrix to CGAffineTransform

Answer 1

Disclaimer

I have never tried this so take it with a grain of salt.

CATRansform3D is a 4x4 matrix which operates on a 3 dimensional homogeneous vector (4x1) to produce another vector of the same type. I am assuming that when rendered, objects described by a 4x1 vector have each element divided by the 4th element and the 3rd element is used only to determine which objects appear on top of which. Assuming this is correct...

Reasoning

The 3x3 matrix returned by findHomography operates on a 2 dimensional homogeneous vector. That process can be thought of in 4 steps

1. The first column of the homography is multiplied by x
2. The second column of the homography is multiplied by y
3. The third column of the homography is multiplied by 1
4. the resulting 1st and 2nd vector elements are divided by the 3rd

You need this process to be replicated in a 4x4 vector in which I am assuming the 3rd element in the resulting vector is meaningless for your purposes.

Solution

Construct your matrix like this (H is your homography matrix)

[H(0,0), H(0,1), 0, H(0,2),
 H(1,0), H(1,1), 0, H(1,2),
      0,      0, 1,      0
 H(2,0), H(2,1), 0, H(2,2)]

This clearly satisfies 1,2 and 3. 4 is satisfied because the homogeneous element is always the last one. That is why the "homogeneous row" if you will had to get bumped down one line. The 1 on the 3rd row is to let the z component of the vector pass through unmolested.

All of the above is done in row major notation (like openCV) to try to keep things from being confusing. You can look at Tommy's answer to see how the conversion to column major looks (you basically just transpose it). Note however that at the moment Tommy and I disagree about how to construct the matrix.

Answer 2

From my reading of the documentation, m11 in CATransform3D is equivalent to a in CGAffineTransform, m12 is equivalent to b and so on.

As per your comment below, I understand the matrix OpenCV returns to be 3x3 (which, in retrospect, is the size you'd expect). So you'd fill in the other elements with those equivalent to the identity matrix. As per Hammer's answer, you want to preserve the portion that deals with the (usually implicit) homogenous coordinate in its place while padding everything else with the identity.

[aside: my original answer was wrong. I've edited it to be correct since I've posted code and Hammer hasn't. This post is marked as community wiki to reflect that it's in no sense solely my answer]

So I think you'd want:

CATransform3D MatToTransform(Mat cvTransform)
{
    CATransform3D transform;

    transform.m11 = cvTransform.at<float>(0, 0);
    transform.m12 = cvTransform.at<float>(1, 0);
    transform.m13 = 0.0f;
    transform.m14 = cvTransform.at<float>(2, 0);

    transform.m21 = cvTransform.at<float>(0, 1);
    transform.m22 = cvTransform.at<float>(1, 1);
    transform.m23 = 0.0f;
    transform.m24 = cvTransform.at<float>(2, 1);

    transform.m31 = 0.0f;
    transform.m32 = 0.0f;
    transform.m33 = 1.0f;
    transform.m34 = 0.0f;

    transform.m41 = cvTransform.at<float>(0, 2);
    transform.m42 = cvTransform.at<float>(1, 2);
    transform.m43 = 0.0f;
    transform.m44 = cvTransform.at<float>(2, 2);

    return transform;
}

Or use cvGetReal1D if you're keeping C++ out of it.

Answer 3

Tommy answer worked for me, but I needed to use double, instead of float. This is also shortened version of the code:

CATransform3D MatToCATransform3D(cv::Mat H) {
    return {
        H.at<double>(0, 0), H.at<double>(1, 0), 0.0, H.at<double>(2, 0),
        H.at<double>(0, 1), H.at<double>(1, 1), 0.0, H.at<double>(2, 1),
        0.0, 0.0, 1.0, 0.0,
        H.at<double>(0, 2), H.at<double>(1, 2), 0.0f, H.at<double>(2, 2)
    };
}