How to remove multiple polygons using Opencv python

Question

Hi StackOverflow team,

I have an image and I want to remove many portions/parts from the image. I tried to use the below code taken from Cropping Concave polygon from Image using Opencv python

Assume I have this image . Also, I have multiple polygons (such as rectangular shapes or any form of a polygon) from the image achieved via lebelme annotation tool. So, I want to remove those shapes from the images or simply changing their pixels to white.

In other words, Labelme Tool will give you a dictionary file, where the dictionary has a key consisting of the points of each portion/polygon/shape)

Then the polygon points can be easily extracted from the dictionary file. After points are extracted, we can define our points by giving names (e.g a,b,s...h), and each one is in this multidimensional format "[[1526, 319], [1526, 376], [1593, 379], [1591, 324]]"

Here I thought of whitening each region. but whitening of multidimensional array seems to be unreliable.

import numpy as np
import cv2
import json

with open('ann1.json') as f:
    data = json.load(f)

#%%
a = data['shapes'][0]['points']; b = data['shapes'][1]['points']; c = data['shapes'][2]['points']; 
#%%
img = cv2.imread("lena.jpg")
pts = np.array(a) # Points

#%%
## (1) Crop the bounding rect
rect = cv2.boundingRect(pts)
x,y,w,h = rect
croped = img[y:y+h, x:x+w].copy()

## (2) make mask
pts = pts - pts.min(axis=0)

mask = np.zeros(croped.shape[:2], np.uint8)
cv2.drawContours(mask, [pts], -1, (255, 255, 255), -1, cv2.LINE_AA)

## (3) do bit-op
dst = cv2.bitwise_and(croped, croped, mask=mask)

## (4) add the white background
bg = np.ones_like(croped, np.uint8)*255
cv2.bitwise_not(bg,bg, mask=mask)
dst2 = bg+ dst

#cv2.imwrite("croped.png", croped)
#cv2.imwrite("mask.png", mask)
#cv2.imwrite("dst.png", dst)
cv2.imwrite("dst2.png", dst2)

Using Lena I have this output . But I need to go further and whiten other points/polygons, for example, the eyes.

As you can see my code can use only one polygon points. I tried appending two other polygon points in my case the two eyes and got .

By appending, I mean I added the multidimensional points (e.g. pts = np.array(a+b+c)).

In short, having an image is there a short way to remove these multiple polygons from the image (by keeping the dimensions of the image) using OpenCV and python.

Json File: https://drive.google.com/file/d/1UyOYUVMHpu2vBBEdR99bwrRX5xIfdOCa/view?usp=sharing

Can you please edit and add example images with the expected output? Currently, the description is not particularly clear. I think you may simply want to combine multiple masks, which a "bitwise or" should achieve. — alkasm, Mar 18 '19 at 21:45
Do you mean you want to loop through every set of points and remove that polygon? — Shawn Mathew, Mar 19 '19 at 19:25
Thanks, @AlexanderReynolds...I have edited the question. let me know if further editing is needed — Jama Hussein Mohamud, Mar 20 '19 at 04:01
@ShawnMathew. thanks.. my intention is to remove every polygon by keeping the dimensions of the image. will looping through the points achieve that? — Jama Hussein Mohamud, Mar 20 '19 at 04:03
@MarkSetchell thanks for your reply the JSON file is attached — Jama Hussein Mohamud, Mar 20 '19 at 11:51
As you are saving as PNG already, rather than make the areas white, would you consider making them transparent instead? You will then be able to differentiate between masked areas and areas that were originally white in the image. — Mark Setchell, Mar 20 '19 at 11:58
@MarkSetchell, thanks, No areas are white in the original images. — Jama Hussein Mohamud, Mar 20 '19 at 12:34

Shawn Mathew · Answer 1 · 2019-03-20T20:26:33.037

1

You'll need to use to loop to go through all the points in the JSON file. I've edited your code to reflect this.

import cv2
import json
import matplotlib.pyplot as plt
import numpy as np

img_path =r"/path/to/lena.png"
json_path = r"/path/to/lena.json"

with open(json_path) as f:
   data = json.load(f)


img = cv2.imread(img_path)

for idx in np.arange(len(data['shapes'])):
    if idx == 0:  #can remove this
        continue  #can remove this
    a = data['shapes'][idx]['points']
    pts = np.array(a) # Points

    ## (1) Crop the bounding rect
    rect = cv2.boundingRect(pts)
    print(rect)
    x,y,w,h = rect
    img[y:y+h, x:x+w] = (255, 255, 255)

    plt.imshow(img)
    plt.show()

Output: I ignored the first line, since it didn't visualize the results nicely. I took your lead and used rectangles instead of polygons. If you need polygons, you'll need to use something like cv2.drawContours() or cv2.polylines() or cv2.fillPoly() as is recommnded in the SO answer you have linked here, to achieve it.

edited Mar 20 '19 at 20:26

answered Mar 20 '19 at 14:59

Shawn Mathew

2,198
1
14
31

Thanks. Your code outputs only the Eye (cropped eye), instead of outputting the whole image without the two eyes. Maybe we need to perform another loop for the image. what do you think? – Jama Hussein Mohamud Mar 20 '19 at 19:26
I thought you'd be working on that part... I can post an update in a bit to answer your question in completeness, I suppose.. – Shawn Mathew Mar 20 '19 at 20:08
Your code only removed the two eyes (Somehow whiten the two eyes). But, If you go back to my question above, I shared three images. The first image is the original image. Then, Using the first polygon point (in my case a) I achieved the second image (which is what I want). then I would like to go further from the second image and remove, for example, eyes anything else that has a polygon shape. I need all this process to happen at the same time by keeping the dimensions of the image the same. As you can see, in my third image I needed to go further but got a messy image. – Jama Hussein Mohamud Mar 21 '19 at 03:41
This code will remove the regions as rectangles, one after the other. If you want polygons, you need to use something like `cv2.fillPoly()` like I mentioned. If you want to maintain the size of the original image, you have to change the color to white. How else would you 'remove' the area? Are you talking about a transparent background like a PNG image? You already have code that does that. Why not just combine them? – Shawn Mathew Mar 21 '19 at 14:45

score 0 · Answer 2 · answered Mar 29 '19 at 13:16

I would like to share with you my expected solution which is a bit modified version of @Shawn Mathew answer.

Input image:

Code:

with open('lena.json') as f:
    json_file = json.load(f)
img = cv2.imread("folder/lena.jpg")
for polygon in np.arange(len(json_file['shapes'])):
    pts = np.array(json_file['shapes'][polygon]['points'])
# If your polygons are rectangular, you can fill with white color to the areas you want be removed by uncommenting the below two lines
#   x,y,w,h = cv2.boundingRect(pts) 
#    cv2.rectangle(img, (x, y), (x+w, y+h), (255, 255, 255), -1)
# if your polygons are different shapes other than rectangles you can just use the below line
    cv2.fillPoly(img, pts =[pts], color=(255,255,255))

plt.imshow(img)
plt.show()

The color of the image changed because of Matplotlib, if you want to preserve the color save the image using cv2.imwrite

You can view the image in its original colors using matplotlib by using `plt.imshow(cv2.cvtColor(result, cv2.COLOR_BGR2RGB));plt.show()`. This is because opencv uses BGR channel ordering whereas everyone else uses RGB — Shawn Mathew, Mar 29 '19 at 14:29

How to remove multiple polygons using Opencv python

2 Answers2