Crop segmented fruit inside a box to get his diameter in pixels

Question

After have segmented my lemons successfully I would like to get his size in pixels and then convert this value to millimeters. I'm reading a thesis were this guys did that but with strawberries. The first step was crop the segmented strawberries in a rectangle:

The image (b) was called the 'minimum rectangle'. According the authors to create it, This is built depending on the extreme values of the region:
- the highest point - the extreme left point - the lowest point - the extreme right point of the region of interest.
Once this is done, the width of the rectangle is measured, which will indicate the measurement of the diameter of the strawberry in pixels.

In my case this is my input image:

And this is my desired output:

I'm programming in python with opencv. I would like to crop my input image and then find the minimum rectangle to get the width of the rectangle which will show the diameter of the lemon in pixels.

According the thesis, to convert the measure in pixels to a measure of the real world as in millimeters, I should take a photography with a rectangle with a 3 cm of side with the same conditions as were take the images of the lemons. Then I should segment this rectangle and then find his minimun rectangle as the image of above and find his width in pixels as result of it with a rectangle of known measures i.g they got 176 pixels of width. Of this way they got:
1mm = 176/30 = 5.87 pixels

With this information I would like to compute the width of my lemons and get this first in pixels, the convert it to milimetters. Guys if you can do it, please suppost that I taked a photography of a know figure of 3cm of side, the same as the thesis. By the moment I can't get the minimun rectangle because I don't know how get it, is because that I asking for his help to you.

Well guys I would like to see your suggestions, any I idea I will apreciate it. Thanks so much.

Thanks you.

Answer 1

Once you have the thresholded image (mask) of your blob of interest (the lemon) it is very straightforward to get its (rotated) minimum area rectangle or bounding rectangle. Use the cv2.minAreaRect function to get the former or the cv2.boundingRect function to get the later. In both cases you need to compute the contours of the binary mask, get the outer and biggest contour and pass that to either function.

Let's see an example for getting both:

# image path
path = "C://opencvImages//"
fileName = "TAkY2.png"

# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)

# Grayscale conversion:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)

# Thresholding:
threshValue, binaryImage = cv2.threshold(grayscaleImage, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)

This is just to get the binary mask, you already have this. This is the result:

Now, get the contours. And just to draw some results, prepare a couple of deep copies of the input that we will use to check out things:

# Find the big contours/blobs on the filtered image:
contours, hierarchy = cv2.findContours(binaryImage, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)

# Deep copies of the input image to draw results:
minRectImage = inputImage.copy()
polyRectImage = inputImage.copy()

Now, get the contours and filter them by a minimum area (minArea) value. You want to just keep the biggest contour - that's the lemon perimeter:

# Look for the outer bounding boxes:
for i, c in enumerate(contours):

    if hierarchy[0][i][3] == -1:

        # Get contour area:
        contourArea = cv2.contourArea(c)
        # Set minimum area threshold:
        minArea = 1000

        # Look for the largest contour:
        if contourArea > minArea:

            # Option 1: Get the minimum area bounding rectangle
            # for this contour:
            boundingRectangle = cv2.minAreaRect(c)
            # Get the rectangle points:
            rectanglePoints = cv2.boxPoints(boundingRectangle)

            # Convert float array to int array:
            rectanglePoints = np.intp(rectanglePoints)
            # Draw the min area rectangle:
            cv2.drawContours(minRectImage, [rectanglePoints], 0, (0, 0, 255), 2)
            cv2.imshow("minAreaRect", minRectImage)
            cv2.waitKey(0)

This portion of code gets you these results. Note that this rectangle is rotated to encompass the minimum area of the contour, just as if you were actually measuring the lemon with a caliper:

You can also get the position of the four corners of this rectangle. Still, inside the loop, we have the following bit of code:

            # Draw the corner points:
            for p in rectanglePoints:
                cv2.circle(minRectImage, (p[0], p[1]), 3, (0, 255, 0), -1)
                cv2.imshow("minAreaRect Points", minRectImage)
                cv2.waitKey(0)

These are the corners of the min area rectangle:

You might or might not like this result. You might be looking for the bounding rectangle that is not rotated. In such case you can use cv2.boundingRect, but first, you need to approximate the contour to a polygon-based set of points. This is the approach, continuing from the last line of code:

        # Option2: Approximate the contour to a polygon:
        contoursPoly = cv2.approxPolyDP(c, 3, True)
        # Convert the polygon to a bounding rectangle:
        boundRect = cv2.boundingRect(contoursPoly)

        # Set the rectangle dimensions:
        rectangleX = boundRect[0]
        rectangleY = boundRect[1]
        rectangleWidth = boundRect[0] + boundRect[2]
        rectangleHeight = boundRect[1] + boundRect[3]

        # Draw the rectangle:
        cv2.rectangle(polyRectImage, (int(rectangleX), int(rectangleY)),
                    (int(rectangleWidth), int(rectangleHeight)), (0, 255, 0), 2)
        cv2.imshow("Poly Rectangle", polyRectImage)
        cv2.imwrite(path + "polyRectImage.png", polyRectImage)
        cv2.waitKey(0)

This is the result:

Edit:

This is the bit that actually crops the lemon from the last image:

# Crop the ROI:
croppedImg = inputImage[rectangleY:rectangleHeight, rectangleX:rectangleWidth]

This is the final output: