Emgucv crop detected shape automatically

Question

I have an application which is going to be used to crop blank spaces from scanned documents for example this image. What I want to do is extract only the card and remove all the white/blank area. I'm using Emgucv FindContours to do this and at the moment I'm able to find the card contour and some noise captured by the scanner in the image as you can see below.

My question is how can I crop the largest contour found or how to extract it by removing other contours and blanks/whitespaces? Or maybe it is possible with the contour index?

Edit: Maybe another possible solution is if is possible to draw the contour to another pictureBox.

Here is the code that I'm using:

Image<Bgr, byte> imgInput;
Image<Bgr, byte> imgCrop;

private void abrirToolStripMenuItem_Click(object sender, EventArgs e)
{
    try
    {
        OpenFileDialog dialog = new OpenFileDialog();

        if (dialog.ShowDialog() ==DialogResult.OK)
        {
            imgInput = new Image<Bgr, byte>(dialog.FileName);
            pictureBox1.Image = imgInput.Bitmap;

            imgCrop = imgInput;
        }
    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.Message);        
    }
}

private void shapeToolStripMenuItem_Click(object sender, EventArgs e)
{
    if (imgCrop == null)
    {
        return;
    }

    try
    {
        var temp = imgCrop.SmoothGaussian(5).Convert<Gray, byte>().ThresholdBinaryInv(new Gray(230), new Gray(255));
        VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint();
        Mat m = new Mat();

        CvInvoke.FindContours(temp, contours, m, Emgu.CV.CvEnum.RetrType.External, Emgu.CV.CvEnum.ChainApproxMethod.ChainApproxSimple);

        for (int i = 0; i < contours.Size; i++)
        {
            double perimeter = CvInvoke.ArcLength(contours[i], true);
            VectorOfPoint approx = new VectorOfPoint();
            CvInvoke.ApproxPolyDP(contours[i], approx, 0.04 * perimeter, true);

            CvInvoke.DrawContours(imgCrop, contours, i, new MCvScalar(0, 0, 255), 2);
            pictureBox2.Image = imgCrop.Bitmap;
        }

    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.Message);
    }
}

Answer 1

I'll give you my answer in C++, but the same operations should be available in Emgu CV.

I propose the following approach: Segment (that is – separate) the target object using the HSV color space. Calculate a binary mask for the object of interest. Get the biggest blob in the binary mask, this should be the card. Compute the bounding box of the card. Crop the card out of the input image

Ok, first get (or read) the input image. Apply a median blur filter, it will help in getting rid of that high-frequency noise (the little grey blobs) that you see on the input. The main parameter to adjust is the size of the kernel (or filter aperture) be careful, though – a high value will result in an aggressive effect and will likely destroy your image:

  //read input image:
  std::string imageName = "C://opencvImages//yoshiButNotYoshi.png";
  cv::Mat imageInput = cv::imread( imageName );

  //apply a median blur filter, the size of the kernel is 5 x 5:
  cv::Mat blurredImage;
  cv::medianBlur ( imageInput, blurredImage, 5 );

This is the result of the blur filter (The embedded image is resized):

Next, segment the image. Exploit the fact that the background is white, and everything else (the object of interest, mainly) has some color information. You can use the HSV color space. First, convert the BGR image into HSV:

  //BGR to HSV conversion:
  cv::Mat hsvImg;
  cv::cvtColor( blurredImage, hsvImg, CV_RGB2HSV );

The HSV color space encodes color information differently than the typical BGR/RGB color space. Its advantage over other color models pretty much depends on the application, but in general, it is more robust while working with hue gradients. I'll try to get an HSV-based binary mask for the object of interest.

In a binary mask, everything you are interested on the input image is colored in white, everything else in black (or vice versa). You can obtain this mask using the inRange function. However, you must specify the color ranges that will be rendered in white (or black) in the output mask. For your image, and using the HSV color model those values are:

  cv::Scalar minColor( 0, 0, 100 ); //the lower range of colors
  cv::Scalar maxColor( 0, 0, 255 ); //the upper range of colors

Now, get the binary mask:

  //prepare the binary mask:
  cv::Mat binaryMask;
  //create the binary mask using the specified range of color
  cv::inRange( hsvImg, minColor, maxColor, binaryMask );
  //invert the mask:
  binaryMask = 255 - binaryMask;

You get this image:

Now, you can get rid of some of the noise (that survived the blur filter) via morphological filtering. Morphological filters are, essentially, logical rules applied on binary (or gray) images. They take a "neighborhood" of pixels in the input and apply logical functions to get an output. They are quite handy while cleaning up binary images. I'll apply a series of logical filters to achieve just that.

I'll first erode the image and then dilate it using 3 iterations. The structuring element is a rectangle of size 3 x 3:

  //apply some morphology the clean the binary mask a little bit:
  cv::Mat SE = cv::getStructuringElement( cv::MORPH_RECT, cv::Size(3, 3) );
  int morphIterations = 3;
  cv::morphologyEx( binaryMask, binaryMask, cv::MORPH_ERODE, SE, cv::Point(-1,-1), morphIterations );
  cv::morphologyEx( binaryMask, binaryMask, cv::MORPH_DILATE, SE, cv::Point(-1,-1), morphIterations );

You get this output. Check out how the noisy blobs are mostly gone:

Now, comes the cool part. You can loop through all the contours in this image and get the biggest of them all. That's a typical operation that I constantly perform, so, I've written a function that does that. It is called findBiggestBlob. I'll present the function later. Check out the result you get after finding and extracting the biggest blob:

  //find the biggest blob in the binary image:
  cv::Mat biggestBlob = findBiggestBlob( binaryMask );

You get this:

Now, you can get the bounding box of the biggest blob using boundingRect:

  //Get the bounding box of the biggest blob:
  cv::Rect bBox = cv::boundingRect( biggestBlob );

Let's draw the bounding box on the input image:

  cv::Mat imageClone = imageInput.clone();
  cv::rectangle( imageClone, bBox, cv::Scalar(255,0,0), 2 );

Finally, let's crop the card out of the input image:

  cv::Mat croppedImage = imageInput( bBox );

This is the cropped output:

This is the code for the findBiggestBlob function. The idea is just to compute all the contours in the binary input, calculate their area and store the contour with the largest area of the bunch:

//Function to get the largest blob in a binary image:
cv::Mat findBiggestBlob( cv::Mat &inputImage ){

    cv::Mat biggestBlob = inputImage.clone();

    int largest_area = 0;
    int largest_contour_index = 0;

    std::vector< std::vector<cv::Point> > contours; // Vector for storing contour
    std::vector< cv::Vec4i > hierarchy;

    // Find the contours in the image
    cv::findContours( biggestBlob, contours, hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE ); 

    for( int i = 0; i < (int)contours.size(); i++ ) {            

        //Find the area of the contour            
        double a = cv::contourArea( contours[i], false);
        //Store the index of largest contour:
        if( a > largest_area ){
            largest_area = a;                
            largest_contour_index = i;
        }

    }

    //Once you get the biggest blob, paint it black:
    cv::Mat tempMat = biggestBlob.clone();
    cv::drawContours( tempMat, contours, largest_contour_index, cv::Scalar(0),
                  CV_FILLED, 8, hierarchy );

    //Erase the smaller blobs:
    biggestBlob = biggestBlob - tempMat;
    tempMat.release();
    return biggestBlob;
}