I'm quite new to OpenCV programming, and I'm developing an app that works like this:
- Take photo of a shelf from camera
- Analyze it with openCV to detect all the rectangles in the photo (i.e. want to find all the products stored on the shelf)
- crop all the rectangle elements found.
My problem is that the detection phase works not so well. The code used to analyze the taken photo:
Bitmap originalPhoto;
byte[] bytes = photo.getByteArray("bitmap");
originalPhoto = BitmapFactory.decodeByteArray(bytes, 0, bytes.length);
Mat imgMat=new Mat();
Utils.bitmapToMat(originalPhoto,imgMat);
Mat imgSource=imgMat.clone();
Imgproc.cvtColor( imgMat, imgMat, Imgproc.COLOR_BGR2GRAY);
Bitmap grayscale=Bitmap.createBitmap(imgMat.cols(),imgMat.rows(),Bitmap.Config.ARGB_8888);
Utils.matToBitmap(imgMat,grayscale);
String root = Environment.getExternalStorageDirectory().toString();
File myDir = new File(root + "/saved_images");
Imgproc.Canny(imgMat,imgMat,0,255);
Bitmap canny=Bitmap.createBitmap(imgMat.cols(),imgMat.rows(),Bitmap.Config.ARGB_8888);
Utils.matToBitmap(imgMat,canny);
Imgproc.GaussianBlur(imgMat, imgMat, new org.opencv.core.Size(1, 1), 2, 2);
Bitmap blur=Bitmap.createBitmap(imgMat.cols(),imgMat.rows(),Bitmap.Config.ARGB_8888);
Utils.matToBitmap(imgMat,blur);
//find the contours
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(imgMat, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
MatOfPoint temp_contour = contours.get(0); //the largest is at the index 0 for starting point
for (int idx = 0; idx < contours.size(); idx++) {
temp_contour = contours.get(idx);
//check if this contour is a square
MatOfPoint2f new_mat = new MatOfPoint2f( temp_contour.toArray() );
int contourSize = (int)temp_contour.total();
MatOfPoint2f approxCurve_temp = new MatOfPoint2f();
Imgproc.approxPolyDP(new_mat, approxCurve_temp, contourSize*0.05, true);
if (approxCurve_temp.total() == 4) {
MatOfPoint points = new MatOfPoint( approxCurve_temp.toArray() );
Rect rect = Imgproc.boundingRect(points);
Core.rectangle(imgSource, new Point(rect.x,rect.y), new Point(rect.x+rect.width,rect.y+rect.height), new Scalar(255, 0, 0, 255), 3);
}
}
Bitmap analyzed=Bitmap.createBitmap(imgSource.cols(),imgSource.rows(),Bitmap.Config.ARGB_8888);
Utils.matToBitmap(imgSource,analyzed);
if(!myDir.exists()) myDir.mkdirs();
fname = "ImageAnalyzed.png";
file = new File (myDir, fname);
if (file.exists ()) file.delete ();
try {
FileOutputStream out = new FileOutputStream(file);
analyzed.compress(Bitmap.CompressFormat.PNG, 100, out);
out.flush();
out.close();
} catch (Exception e) {
e.printStackTrace();
}
The result photo is not correctly analyzed, it returns to me a photo like this:
Has anyone an idea on how to solve it?
Thanks in advance, Fabio
edit
I found that using ImgProc.APPROX_CHAIN_NONE instead of ImgProc.APPROX_CHAIN_SIMPLE gives me much better results... Now I have to tune canny thresholds to give the program independency wrt the type of image I analyze. I think that using mean or median values of the image will give me better results as well.