OpenCV Rotation (Deskewing) in Android - C++ to Java Conversion

Question

I'm trying to detect the skew angle of a text in an image link. The problem is that this post is written in C++, and I'm having trouble converting some things to Java.

I did the implementation of Hugh Transform. Most of the conversion to java I did guiding me from this post (1). But it's not working fine. Is giving an angle of 0.27919363 when is supposed to give an angle of 15.9882. This is the image I'm working with.

And this is my code:

public double compute_skew1(String filename){
    Log.d(TAG, "Computing skew 1");

    Mat src = Highgui.imread(filename, 0);
    Size size = src.size();
    //double minLineSize = 20;
    double minLineSize = src.width() / 2.f;

    Core.bitwise_not(src, src);

    Mat lines = new Mat();
    double angle = 0.;
    try {
        Imgproc.HoughLinesP(src, lines, 1, Math.PI / 180, 100, minLineSize, 20);

        Mat disp_lines = new Mat(size, CvType.CV_8UC1, new Scalar(0, 0, 0));
        int nb_lines = lines.cols();
        for (int i = 0; i < nb_lines; i++) {
            double[] vec = lines.get(0, i);
            double x1 = vec[0], 
                   y1 = vec[1],
                   x2 = vec[2],
                   y2 = vec[3];
            Point start = new Point(x1, y1);
            Point end = new Point(x2, y2);
            Core.line(disp_lines, start, end, new Scalar(255,0,0));
            angle += Math.atan2(y2 - y1, x2 - x1);
        }
        angle /= nb_lines; // mean angle, in radians.*/
        //Log.d(TAG, "ANGLE: "+angle);
        Log.d(TAG, "ANGLE: "+ angle * 180 / Math.PI);
    } catch (Exception e) {
        Log.e(TAG, "Error in compute_skew1");
        Log.e(TAG, e.getMessage());
    }

    return angle;
}

I'm almost certain the problem is with this line "int nb_lines = lines.cols();" since the original line is "unsigned nb_lines = lines.size();", java doesn't jave unsigned variables, and that's the way it works in the post(1). Also, I don't quite understand this line "double[] vec = lines.get(0, i);" but that's the way it worked in the post (1) as well. What am I doing wrong?

Also, after I get the angle, I need to do the rotation or deskewing of the text, and I also have some trouble with this conversion, especially with this part of the code:

std::vector<cv::Point> points;
cv::Mat_<uchar>::iterator it = img.begin<uchar>();
cv::Mat_<uchar>::iterator end = img.end<uchar>();
for (; it != end; ++it)
if (*it)
  points.push_back(it.pos());

With the help of this post2, I believed this to be the conversion:

List <Point> points = new ArrayList<Point>();
for (int i = 0; i < img.rows(); i++) {
    for (int j = 0; j < img.cols(); j++) {
        double pixel = img.get(i, j)[0];
        if (pixel != 0.0)
            points.add(new Point(i,j));
    }
}

But does not work, there is never a pixel = 0.0 and so the p array just fills with every pixel.

So. Please let me know what I'm doing wrong. Thanks in advance.

I've resolved the first issue. The problem was in the minLineSize, I was using 20 instead of half the image's width. Also, the reason I was seeing so much different results in the angles is because I was printing them in radians. (See commented lines) NOTE: Still have not resolved second issue. — Mauricio, Aug 19 '14 at 23:52
No, I decided to stick with C++ and use the Android NDK to add it to my app, since most of the OpenCV documentation and sample codes are in C++. @DarkLink — Mauricio, May 06 '15 at 21:40

bodoroman · Answer 1 · 2016-01-20T14:30:15.993

6

Here is the java code to calculate skew angle:

    Mat source = Imgcodecs.imread(input.getName(),0);
    Size size = source.size();
    Core.bitwise_not(source, source);
    Mat lines = new Mat();
    Imgproc.HoughLinesP(source, lines, 1, Math.PI / 180, 100, size.width / 2.f, 20);
    double angle = 0.;
    for(int i = 0; i<lines.height(); i++){
        for(int j = 0; j<lines.width();j++){
            angle += Math.atan2(lines.get(i, j)[3] - lines.get(i, j)[1], lines.get(i, j)[2] - lines.get(i, j)[0]);
        }
    }
    angle /= lines.size().area();
    angle = angle * 180 / Math.PI;

edited Jan 20 '16 at 14:30

answered Jan 20 '16 at 14:12

bodoroman

278
2
16

I assume you are downvoted, because your code is not working. E.g. lines is a matrix => you need to do a height on Size not on matrix. – cilap Feb 12 '18 at 06:21
could you please comment your angle calculation. What is contained in the lines.get(i,j)[arrayIndex]? Also what means index 0 to 3? – cilap Feb 13 '18 at 09:11

score 2 · Answer 2 · edited Aug 23 '16 at 06:19

here is the java code to deskew your image:

Mat deskew(Mat src, double angle) {
    Point center = new Point(src.width()/2, src.height()/2);
    Mat rotImage = Imgproc.getRotationMatrix2D(center, angle, 1.0);
    //1.0 means 100 % scale
    Size size = new Size(src.width(), src.height());
    Imgproc.warpAffine(src, src, rotImage, size, Imgproc.INTER_LINEAR + Imgproc.CV_WARP_FILL_OUTLIERS);
    return src;
}

The angle should be in radians not in degrees.

score 1 · Answer 3 · answered May 08 '18 at 11:02

1

This is possible using classes of tess4j i.e ImageDeskew & ImageHelper

ImageDeskew imgdeskew=new ImageDeskew(img); // BufferedImage img
ImageHelper.rotateImage(bim, -imgdeskew.getSkewAngle()); // rotateImage static method

answered May 08 '18 at 11:02

Bittu Choudhary

65
6

score 0 · Answer 4 · answered Sep 09 '21 at 13:24

private fun main(){
    val bmp:Bitmap? = null //Any bitmap (if you are working with bitmap)
    var mRgba = Mat() // else you can direct use MAT on onCameraFrame 
    val mGray = Mat()
    val bmp32: Bitmap = bmp.copy(Bitmap.Config.ARGB_8888, true)
    Utils.bitmapToMat(bmp32, mRgba)
    Imgproc.cvtColor(mRgba, mGray, Imgproc.COLOR_BGR2GRAY)
    mRgba = makeOrientationCorrection(mRgba,mGray)// here actual magic starts
    Imgproc.cvtColor(mRgba, mGray, Imgproc.COLOR_BGR2GRAY)
    val bmpOutX = Bitmap.createBitmap(
        mRgba.cols(),
        mRgba.rows(),
        Bitmap.Config.ARGB_8888
    )
    Utils.matToBitmap(mRgba, bmpOutX)
    binding.imagePreview.setImageBitmap(bmpOutX!!)
}

private fun makeOrientationCorrection(mRGBA:Mat, mGRAY:Mat):Mat{
    val dst = Mat()
    val cdst = Mat()
    val cdstP: Mat
    Imgproc.Canny(mGRAY, dst, 50.0, 200.0, 3, false)
    Imgproc.cvtColor(dst, cdst, Imgproc.COLOR_GRAY2BGR)
    cdstP = cdst.clone()

    val linesP = Mat()
    Imgproc.HoughLinesP(dst, linesP, 1.0, Math.PI/180, 50, 50.0, 10.0)

    var biggestLineX1 = 0.0
    var biggestLineY1 = 0.0
    var biggestLineX2 = 0.0
    var biggestLineY2 = 0.0
    var biggestLine = 0.0
    for (x in 0 until linesP.rows()) {
        val l = linesP[x, 0]

        Imgproc.line(
            cdstP, org.opencv.core.Point(l[0], l[1]),
            org.opencv.core.Point(l[2], l[3]),
            Scalar(0.0, 0.0, 255.0), 3, Imgproc.LINE_AA, 0)
    }

    for (x in 0 until linesP.rows()) {
        val l = linesP[x, 0]
        val x1 = l[0]
        val y1 = l[1]
        val x2 = l[2]
        val y2 = l[3]
        val lineHeight = sqrt(((x2 - x1).pow(2.0)) + ((y2 - y1).pow(2.0)))
        if(biggestLine<lineHeight){
            val angleOfRotationX1 = angleOf(PointF(x1.toFloat(),y1.toFloat()),PointF(x2.toFloat(),y2.toFloat()))
            Log.e("angleOfRotationX1","$angleOfRotationX1")
            if(angleOfRotationX1<45.0 || angleOfRotationX1>270.0){
                biggestLine = lineHeight
                if(angleOfRotationX1<45.0){
                    biggestLineX1 = x1
                    biggestLineY1 = y1
                    biggestLineX2 = x2
                    biggestLineY2 = y2
                }
                if(angleOfRotationX1>270.0){
                    biggestLineX1 = x2
                    biggestLineY1 = y2
                    biggestLineX2 = x1
                    biggestLineY2 = y1
                }
            }
        }
        if(x==linesP.rows()-1){
            Imgproc.line(
                cdstP, org.opencv.core.Point(biggestLineX1, biggestLineY1),
                org.opencv.core.Point(biggestLineX2, biggestLineY2),
                Scalar(255.0, 0.0, 0.0), 3, Imgproc.LINE_AA, 0)
        }
    }
    var angle = angleOf(PointF(biggestLineX1.toFloat(),biggestLineY1.toFloat()),PointF(biggestLineX2.toFloat(),biggestLineY2.toFloat()))
    Log.e("angleOfRotationX2","$angle")
    angle -= (angle * 2)
    return deskew(mRGBA,angle)
}

fun angleOf(p1: PointF, p2: PointF): Double {
    val deltaY = (p1.y - p2.y).toDouble()
    val deltaX = (p2.x - p1.x).toDouble()
    val result = Math.toDegrees(Math.atan2(deltaY, deltaX))
    return if (result < 0) 360.0 + result else result
}

private fun deskew(src:Mat, angle:Double):Mat{
    val center = org.opencv.core.Point((src.width() / 2).toDouble(), (src.height() / 2).toDouble())
    val scaleBy = if(angle<0){
        1.0+((0.5*angle)/45)//max scale down by 0.50(50%) based on angle
    }else{
        1.0-((0.3*angle)/45)//max scale down by 0.50(50%) based on angle
    }

    Log.e("scaleBy",""+scaleBy)
    val rotImage = Imgproc.getRotationMatrix2D(center, angle, scaleBy)
    val size = Size(src.width().toDouble(), src.height().toDouble())
    Imgproc.warpAffine(src, src, rotImage, size, Imgproc.INTER_LINEAR + Imgproc.CV_WARP_FILL_OUTLIERS)
    return src
}

Make sure you run this "makeOrientationCorrection()" method on another thread. otherwise, UI won't update for 2-5 sec.

OpenCV Rotation (Deskewing) in Android - C++ to Java Conversion

4 Answers4