How to crop an image from screenshot with the use of OpenCV?

Question

I recently began studying image processing and took a task where I need to crop an image from mobile Instagram screenshot via use of OpenCV. I need to find edges of the image with contours and crop, but I'm not sure how to do this correctly.

I've tried to look up some examples like these:

How to crop biggest rectangle out of an image

https://www.quora.com/How-can-I-detect-an-object-from-static-image-and-crop-it-from-the-image-using-openCV

How to detect edge and crop an image in Python

How to crop rectangular shapes in an image using Python

But I'm still don't understand how to do it in my case.

Basically I have images like these:

https://i.stack.imgur.com/hie3V.jpg and https://i.stack.imgur.com/PksBc.jpg

And the result should be like this:

https://i.stack.imgur.com/fkQhW.jpg

https://i.stack.imgur.com/Nnn21.jpg

Screenshots used need to be only from mobile version of Instagram and it can be assumed that they are always of rectangular shape

And if there are more than one image like here:

https://i.stack.imgur.com/hj1Eh.jpg

Then only one of the two is cropped (which one doesn't matter). For example:

https://i.stack.imgur.com/jZN81.jpg

Thanks!

Answer 1

One of the prominent feature in your snapshot images is the white background color. Everything appears on top of it, even that user image. So we will try to segment out the background which would leave us with smaller components such as Instagram icon, likes, etc. Then we will pick the largest element assuming that the user image is the largest element present on the screen. Then we will simply find the cv2.boundingRect() of the largest contour and crop the snapshot accordingly as:

import cv2
import numpy as np

img = cv2.imread("/path/to/img.jpg")

white_lower = np.asarray([230, 230, 230])
white_upper = np.asarray([255, 255, 255])

mask = cv2.inRange(img, white_lower, white_upper)
mask = cv2.bitwise_not(mask)

Now we fill find contours in this mask and select the largest one.

im, cnt, hierarchy = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
largest_contour = max(cnt, key=lambda x:cv2.contourArea(x))
bounding_rect = cv2.boundingRect(largest_contour)

cropped_image = img[bounding_rect[1]: bounding_rect[1]+bounding_rect[3],
                bounding_rect[0]:bounding_rect[0]+bounding_rect[2]]