Can't get OpenCV's warpPerspective to work on Android

Question

I've been struggling to implement a quad to quad system in my Android application. The aim is to let the user take a picture, add 4 cornerpoints and have that quad extracted from the image as a rectangle.

I had a look at this method and this question to use OpenCV for this. The resulting code is this:

public static Bitmap warp(Bitmap image, MyPoint p1, MyPoint p2, MyPoint p3, MyPoint p4) {
    int resultWidth = 500;
    int resultHeight = 500;

    Mat inputMat = new Mat(image.getHeight(), image.getHeight(), CvType.CV_8UC4);
    Utils.bitmapToMat(image, inputMat);
    Mat outputMat = new Mat(resultWidth, resultHeight, CvType.CV_8UC4);

    Point ocvPIn1 = new Point(p1.getX(), p1.getY());
    Point ocvPIn2 = new Point(p2.getX(), p2.getY());
    Point ocvPIn3 = new Point(p3.getX(), p3.getY());
    Point ocvPIn4 = new Point(p4.getX(), p4.getY());
    List<Point> source = new ArrayList<Point>();
    source.add(ocvPIn1);
    source.add(ocvPIn2);
    source.add(ocvPIn3);
    source.add(ocvPIn4);
    Mat startM = Converters.vector_Point2f_to_Mat(source);

    Point ocvPOut1 = new Point(0, 0);
    Point ocvPOut2 = new Point(0, resultHeight);
    Point ocvPOut3 = new Point(resultWidth, resultHeight);
    Point ocvPOut4 = new Point(resultWidth, 0);
    List<Point> dest = new ArrayList<Point>();
    dest.add(ocvPOut1);
    dest.add(ocvPOut2);
    dest.add(ocvPOut3);
    dest.add(ocvPOut4);
    Mat endM = Converters.vector_Point2f_to_Mat(dest);      

    Mat perspectiveTransform = new Mat(3, 3, CvType.CV_32FC1);
    Core.perspectiveTransform(startM, endM, perspectiveTransform);

    Imgproc.warpPerspective(inputMat, 
                            outputMat,
                            perspectiveTransform,
                            new Size(resultWidth, resultHeight), 
                            Imgproc.INTER_CUBIC);

    Bitmap output = Bitmap.createBitmap(resultWidth, resultHeight, Bitmap.Config.RGB_565);
    Utils.matToBitmap(outputMat, output);
    return output;
}

While testing, I make sure the order of the corner points is top-left, bottom-left, bottom-right, top-right.

The strange thing is that the result isn't always the same. Most of the times it shows a square of a single color, sometimes a black square, sometimes a diagonal line with different colors in it. Even experimenting with startM = endM results in non-deterministic behaviour.

What am I missing here?

this works some how but, some times it gives mirror and/or rotated copy of the final bitmap. ? is this desired out come — Qadir Hussain, May 11 '15 at 10:45

score 5 · Accepted Answer · answered Jun 28 '13 at 11:20

5

Found it, the problem was in these lines:

Mat perspectiveTransform = new Mat(3, 3, CvType.CV_32FC1);
Core.perspectiveTransform(startM, endM, perspectiveTransform);

Which should be replaced by this:

Mat perspectiveTransform = Imgproc.getPerspectiveTransform(startM, endM);

answered Jun 28 '13 at 11:20

gleerman

1,793
4
24
38

1

Thanks @Aegoins u saved a lot of my efforts. – DearDhruv Sep 27 '13 at 05:33
Hi there, I am following the same thing as you but I am getting a weird output. Actually my input is a business card and using the perspective transform I want to extract just the card from the image. The output image that I am getting is just a blank colored image which has the color from one of the colors in the card. – Manpreet Nov 09 '14 at 22:08
@Aegonis this works some how but, some times it gives mirror copy of the final bitmap. ? is this desired out come – Qadir Hussain Apr 15 '15 at 10:53
@Manpreet see my answer's code and check if it solves your problem. – jonathanrz Jul 31 '15 at 21:51

score 2 · Answer 2 · answered Jul 31 '15 at 21:50

I had some problems with your code, so I made changes until I get a working version, here is my code if someone had problems with the question code:

Bitmap warp(Bitmap image, Point topLeft, Point topRight, Point bottomLeft, Point bottomRight) {
    int resultWidth = (int)(topRight.x - topLeft.x);
    int bottomWidth = (int)(bottomRight.x - bottomLeft.x);
    if(bottomWidth > resultWidth)
        resultWidth = bottomWidth;

    int resultHeight = (int)(bottomLeft.y - topLeft.y);
    int bottomHeight = (int)(bottomRight.y - topRight.y);
    if(bottomHeight > resultHeight)
        resultHeight = bottomHeight;

    Mat inputMat = new Mat(image.getHeight(), image.getHeight(), CvType.CV_8UC1);
    Utils.bitmapToMat(image, inputMat);
    Mat outputMat = new Mat(resultWidth, resultHeight, CvType.CV_8UC1);

    List<Point> source = new ArrayList<>();
    source.add(topLeft);
    source.add(topRight);
    source.add(bottomLeft);
    source.add(bottomRight);
    Mat startM = Converters.vector_Point2f_to_Mat(source);

    Point ocvPOut1 = new Point(0, 0);
    Point ocvPOut2 = new Point(resultWidth, 0);
    Point ocvPOut3 = new Point(0, resultHeight);
    Point ocvPOut4 = new Point(resultWidth, resultHeight);
    List<Point> dest = new ArrayList<>();
    dest.add(ocvPOut1);
    dest.add(ocvPOut2);
    dest.add(ocvPOut3);
    dest.add(ocvPOut4);
    Mat endM = Converters.vector_Point2f_to_Mat(dest);

    Mat perspectiveTransform = Imgproc.getPerspectiveTransform(startM, endM);

    Imgproc.warpPerspective(inputMat, outputMat, perspectiveTransform, new Size(resultWidth, resultHeight));

    Bitmap output = Bitmap.createBitmap(resultWidth, resultHeight, Bitmap.Config.ARGB_8888);
    Utils.matToBitmap(outputMat, output);
    return output;
}

@jonathanrz You've set height for width also in this line Mat inputMat = new Mat(image.getHeight(), image.getHeight(), CvType.CV_8UC1); — Kashif Anwaar, Apr 27 '19 at 15:08
exactly what i was looking for!! 1 million thanks to @jonathanrz — Sjd, Jan 14 '20 at 12:45

Can't get OpenCV's warpPerspective to work on Android

2 Answers2

Linked