1

I'm quite new to OpenCV programming, and I'm developing an app that works like this:

  1. Take photo of a shelf from camera
  2. Analyze it with openCV to detect all the rectangles in the photo (i.e. want to find all the products stored on the shelf)
  3. crop all the rectangle elements found.

My problem is that the detection phase works not so well. The code used to analyze the taken photo:

Bitmap originalPhoto;
        byte[] bytes = photo.getByteArray("bitmap");

        originalPhoto = BitmapFactory.decodeByteArray(bytes, 0, bytes.length);

            Mat imgMat=new Mat();

            Utils.bitmapToMat(originalPhoto,imgMat);

        Mat imgSource=imgMat.clone();

        Imgproc.cvtColor( imgMat, imgMat, Imgproc.COLOR_BGR2GRAY);
        Bitmap grayscale=Bitmap.createBitmap(imgMat.cols(),imgMat.rows(),Bitmap.Config.ARGB_8888);
        Utils.matToBitmap(imgMat,grayscale);

        String root = Environment.getExternalStorageDirectory().toString();
        File myDir = new File(root + "/saved_images");

        Imgproc.Canny(imgMat,imgMat,0,255);
        Bitmap canny=Bitmap.createBitmap(imgMat.cols(),imgMat.rows(),Bitmap.Config.ARGB_8888);
        Utils.matToBitmap(imgMat,canny);

        Imgproc.GaussianBlur(imgMat, imgMat, new  org.opencv.core.Size(1, 1), 2, 2);
        Bitmap blur=Bitmap.createBitmap(imgMat.cols(),imgMat.rows(),Bitmap.Config.ARGB_8888);
        Utils.matToBitmap(imgMat,blur);

        //find the contours
        List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
        Imgproc.findContours(imgMat, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);

        MatOfPoint temp_contour = contours.get(0); //the largest is at the index 0 for starting point

        for (int idx = 0; idx < contours.size(); idx++) {
            temp_contour = contours.get(idx);
                //check if this contour is a square

                MatOfPoint2f new_mat = new MatOfPoint2f( temp_contour.toArray() );

                int contourSize = (int)temp_contour.total();
                MatOfPoint2f approxCurve_temp = new MatOfPoint2f();
                Imgproc.approxPolyDP(new_mat, approxCurve_temp, contourSize*0.05, true);

                if (approxCurve_temp.total() == 4) {
                    MatOfPoint points = new MatOfPoint( approxCurve_temp.toArray() );
                    Rect rect = Imgproc.boundingRect(points);
                    Core.rectangle(imgSource, new Point(rect.x,rect.y), new Point(rect.x+rect.width,rect.y+rect.height), new Scalar(255, 0, 0, 255), 3);

                }

        }
        Bitmap analyzed=Bitmap.createBitmap(imgSource.cols(),imgSource.rows(),Bitmap.Config.ARGB_8888);
        Utils.matToBitmap(imgSource,analyzed);


        if(!myDir.exists()) myDir.mkdirs();

        fname = "ImageAnalyzed.png";
        file = new File (myDir, fname);
        if (file.exists ()) file.delete ();
        try {
            FileOutputStream out = new FileOutputStream(file);
            analyzed.compress(Bitmap.CompressFormat.PNG, 100, out);
            out.flush();
            out.close();

        } catch (Exception e) {
            e.printStackTrace();
        }

The result photo is not correctly analyzed, it returns to me a photo like this: instead of 16 rectangles, I obtain other

Has anyone an idea on how to solve it?

Thanks in advance, Fabio

edit

I found that using ImgProc.APPROX_CHAIN_NONE instead of ImgProc.APPROX_CHAIN_SIMPLE gives me much better results... Now I have to tune canny thresholds to give the program independency wrt the type of image I analyze. I think that using mean or median values of the image will give me better results as well.

panc_fab
  • 522
  • 1
  • 7
  • 24
  • http://stackoverflow.com/a/30096768/2987421 – Santosh Kathait Aug 24 '15 at 10:22
  • 1
    You may need to set up the thresholds of the Canny detector. Did you try other values ? As well, you should blur before canny edge detection. – xiawi Aug 24 '15 at 10:23
  • thanks for the tip. Yes, I've tried different threshold values but did not affect the result. Later I'll try to swap blur and canny order. thank you – panc_fab Aug 24 '15 at 10:33
  • 1
    please have a look at your canny image and/or your contours, I bet your desired rectangles aren't visible at all... – Micka Aug 24 '15 at 11:08
  • My canny image shows b/w rectangles well... I save all images aftwr each process and all works well, but not the red rects – panc_fab Aug 24 '15 at 12:12
  • panc_fab did u find the solution. Actually i also need that functionality. if u find the solution. so can you help me please – UltimateDevil Jun 17 '17 at 10:24
  • @VikasKumarTiwari please read also one onf my questions on opencv q&a: http://answers.opencv.org/question/85884/detect-boxes-on-shelf-android-opencv/ I've managed to use also that method, combined with this, to get a working (thus non optimal) solution. I do not remember but I probably have the source code, if I'll find I'll let u know and maybe share with u – panc_fab Jun 19 '17 at 05:50

0 Answers0