Object detection and segmentation Using Python

Question

I am an undergraduate student. I am new to image processing and python.

I have many images of plants samples and their description(called labels which are stuck on the sample) as shown in the below Figure. I need to Automatically segment only those labels from the sample.

I tried thresholding based on colour, but it failed. Could you please suggest me an example to do this task. I need some ideas or codes to make it completely automatic segmentation.

Please help me if you are experts in image processing and Python, I need your help to complete this task.

The rectangle is detected on the Top Left, but it should be on bottom right. Could you please tell me where is my mistake and how to correct it. I have also given the code below.

score 0 · Answer 1 · answered Aug 15 '16 at 14:30

You can try a template matching with a big white rectangle to identify the area where information is stored.

http://docs.opencv.org/3.1.0/d4/dc6/tutorial_py_template_matching.html#gsc.tab=0

When it will be done, you will be able to recognize characters in this area... You save a small subimage, and with a tool like pytesseract you will be able to read characters.

https://pypi.python.org/pypi/pytesseract

You have other OCR here with some examples : https://saxenarajat99.wordpress.com/2014/10/04/optical-character-recognition-in-python/

Good luck !

Hi @FrECM, I have changes made on template matching. But I dont know where I went wrong. — 034568, Aug 16 '16 at 10:41

score 0 · Answer 2 · edited May 23 '17 at 12:08

Why using color threshold? I tried this one with ImageJ and get nice results. I just converted the image to 8bit and binarise using a fixed threshold (166 in this case). You can choose the best threshold from the image histogram. Then you just need to find your white rectangle region and read the characters like FrsECM suggested.

Here's an example in c++:

#include "opencv2/imgproc.hpp"
#include "opencv2/highgui.hpp"
#include <stdlib.h>
#include <stdio.h>

using namespace cv;

/// Global variables
int threshold_nvalue = 166; 
const int thresh_increment = 2;
int threshold_type = THRESH_BINARY;//1
int const max_value = 255;
int const morph_size = 3;
int const  min_blob_size = 1000;
Mat src, src_resized, src_gray, src_thresh, src_morph;

/**
* @function main
*/
int main(int argc, char** argv)
{
    /// Load an image
    src = imread("C:\\Users\\phili\\Pictures\\blatt.jpg", 1);
    //Resize for displaying it properly
    resize(src, src_resized, Size(600, 968));
    /// Convert the image to Gray
    cvtColor(src_resized, src_gray, COLOR_RGB2GRAY);    
    /// Region of interest
    Rect label_rect;
    //Binarization sing fixed threshold
    threshold(src_gray,src_thresh, thres, max_value, threshold_type);
    //Erase small object using morphologie
    Mat element = getStructuringElement(0, Size(2 * morph_size + 1, 2 * morph_size + 1), Point(morph_size, morph_size));
    morphologyEx(src_thresh, src_morph, MORPH_CLOSE, element);  

    //find white objects and their contours
    std::vector<std::vector<Point> > contours;
    std::vector<Vec4i> hierarchy;
    findContours(src_morph, contours, CV_RETR_TREE, CV_CHAIN_APPROX_NONE, Point(0, 0));
    for (std::vector<std::vector<Point> >::iterator it = contours.begin(); it != contours.end(); ++it)
    {

        //just big blobs
        if (it->size()>min_blob_size)
        {
            //approx contour and check for rectangle
            std::vector<Point> approx;
            approxPolyDP(*it, approx, 0.01*arcLength(*it, true), true);
            if (approx.size() == 4)
            {
                //just for visualization 
                drawContours(src_resized, approx, 0, Scalar(0, 255, 255),-1);
                //bounding rect for ROI
                label_rect = boundingRect(approx);
                //exit loop             
                break;
            }
        }


    }
    //Region of interest
    Mat label_roi = src_resized(label_rect);

    //OCR comes here...
}

[Here](http://stackoverflow.com/questions/11424002/how-to-detect-simple-geometric-shapes-using-opencv) you can read how to detect simple shapes like an rectangle — PSchn, Aug 16 '16 at 10:29
Thanks for the information @PSchn, I have updated the result based on template matching but I dont know where I went wrong. — 034568, Aug 16 '16 at 10:42
I think you mixed two approaches. Do a binarization and a contour finding to detect your rectangle or the template matching routine dont mix. I suggest to do the binarization, find your contours, clean them up, approx a polygon,.. and there you go. I you use template matching, then forget about the binarisation. Has the template the same size as your label? — PSchn, Aug 16 '16 at 10:58

Object detection and segmentation Using Python

2 Answers2