How do I crop an image based on custom mask

Question

I start my model and get a mask prediction of the area I would like to crop.

img = cv2.imread('picture.jpg')
img = cv2.resize(img, (224, 224))
print(img.shape)
T = np.zeros((1, 224, 224, 3), dtype='float32')
T[0]=img
prediction = model.predict(T , verbose=1)
prediction = prediction[0, :, : , :]
plt.imshow(img)
plt.show()
print(prediction.shape)
plt.imshow(np.squeeze(prediction).astype(np.float32))
plt.show()

This code is the closest I have gotten to the desired image, the problem with this image that is provided is that it doesn't crop out black background

result = prediction * img
plt.imshow((result/255).astype(np.float32))
plt.show()

My desired image should be like this

what do you expect, instead of blackness? that is exactly how multiplication works. — Christoph Rackwitz, Aug 31 '22 at 23:28
so you want it *cropped* closely? okay, call `boundingRect`, on the mask `prediction > 0.5` (assuming it's scaled like that), then take these values and do a numpy slice — Christoph Rackwitz, Sep 01 '22 at 06:39

PepeChuy · Accepted Answer · 2022-09-01T11:07:05.233

1

You could try this:

prediction = np.squeeze(prediction).astype(np.float32)
threshold = 0.5
xmin,ymin,xmax,ymax = (-1,-1,-1,-1)
for j in range(224):
    for i in range(224):
        if prediction[j,i] <= threshold: continue
        xmin = i if xmin == -1 or xmin > i else xmin
        xmax = i if xmax == -1 or xmax < i else xmax
        ymin = j if ymin == -1 or ymin > j else ymin
        ymax = j if ymax == -1 or ymax < j else ymax
cropImg = img[ymin:ymax,xmin:xmax]

edited Sep 01 '22 at 11:07

answered Aug 31 '22 at 23:54

PepeChuy

84
3

The output is same as plt.imshow(img) plt.show() – LuQchor Sep 01 '22 at 00:49
You could modify the `threshold` value in order to get the wanted bounding box. – PepeChuy Sep 01 '22 at 11:09

score 0 · Answer 2 · answered Sep 01 '22 at 12:01

You can try to find the bounding box around your predicted mask and then crop your image using the detected bounding box. Using the code from this excellent SO answer, it would result in something like this:

def bbox2(img):
    # from https://stackoverflow.com/a/31402351/19249364
    rows = np.any(img, axis=1)
    cols = np.any(img, axis=0)
    rmin, rmax = np.where(rows)[0][[0, -1]]
    cmin, cmax = np.where(cols)[0][[0, -1]]
    return rmin, rmax, cmin, cmax

bbox = bbox2(prediction.squeeze())
cropped = img[bbox[0]:bbox[1]+1, bbox[2]:bbox[3]+1, :]

Couldn't test this code, so just tell me if it doesn't work!

The output image is same as plt.imshow(img), but my problem is solved with code above,than you for your help. — LuQchor, Sep 01 '22 at 12:49

How do I crop an image based on custom mask

2 Answers2