0

I am training my model with several images.

When training my model I realized that I could increase my accuracy by replacing the zero elements in my image array with other values and so I replaced them with the median value of my image as shown with the following code.

import cv2
import imutils
import numpy as np

r_val_all = np.zeros((2000,112,112))
for r in range(len(r_val)): 
    #LOAD IMAGES
    r_image_v = cv2.imread(r_val[r])
    r_gray_v = cv2.cvtColor(r_image_v, cv2.COLOR_BGR2GRAY)
    r_gray_v = imutils.resize(r_gray_v, width=112, height=112)
    n = np.median(r_gray_v[r_gray_v > 0])
    r_gray_v[r_gray_v == 0] = n
    r_val_all[r,:,:] = r_gray_v

The accuracy did improve however it is not quite there yet.

What I actually require is something where the zero elements are replaced with a continuation of the pre-existent array values.

However I was not sure how to tackle such a problem are there any tools that perform the operation I require?

Justin
  • 155
  • 1
  • 7
  • Does this help you? https://stackoverflow.com/questions/3662361/fill-in-missing-values-with-nearest-neighbour-in-python-numpy-masked-arrays – K.Cl Apr 01 '21 at 21:41
  • I read what you sent me and it seemed as if it was exactly what I needed but I implemented it for my code and I ended up with the same result as before. Not sure how I can attach the code I implemented in the comments as it would be too many characters – Justin Apr 01 '21 at 22:32
  • You can upload it to github for example and link it. – K.Cl Apr 02 '21 at 03:12
  • https://github.com/SchJas/headpose/issues/1 - pls note that I tried to implement a code that best fits with the rest of my code as the one provided on the link was selecting the array values they required to change and I believe they made them null, whereas I tried to convert my zero elements to nan and then apply nearest neighbor – Justin Apr 02 '21 at 13:32

1 Answers1

0

I used the second answer from the link, tell me if this is close to what you want, because it appeared to be what you wanted.

Creating one sample image and center it, so it's somewhat close to your first example image.

import numpy as np
import matplotlib.pyplot as plt
image = np.zeros((100, 100))
center_noise = np.random.normal(loc=10, size=(50, 50))
image[25:75, 25:75] = center_noise
plt.imshow(image, cmap='gray')

Output

Inspired by rr_gray = np.where(rr_gray==0, np.nan, rr_gray) #convert zero elements to nan in your code, I'm replacing the zeros with NaN.

image_centered = np.where(image == 0, np.nan, image)
plt.imshow(image_centered, cmap='gray')

Output

Now I used the function in the second answer of the link, fill.

test = fill(image_centered)
plt.imshow(test, cmap='gray')

This is the result

With fill

I'm sorry I can't help you more. I wish I could, I'm just not very well versed in image processing. I looked at your code and couldn't figure out why it's not working, sorry.

K.Cl
  • 1,615
  • 2
  • 9
  • 18
  • Sorry, I placed the last code snippet now. The `fill` function is from here: https://stackoverflow.com/a/27745627/8310295 – K.Cl Apr 02 '21 at 16:14
  • In that repository you created, can you upload the image you're testing and exactly the code you used on that image that resulted on it not working? I will take a last look. – K.Cl Apr 02 '21 at 16:29
  • I uploaded one image from the dataset that I am using on github. From my code all you need to do is remove the first for loop and apply this code instead to upload the image: `rr_image = cv2.imread('AFW_79378097_3_0.jpg')` – Justin Apr 02 '21 at 17:27
  • I uploaded the updated code on the repository, sorry did not realize you requested it as well. – Justin Apr 02 '21 at 17:53
  • I reproduced the result from your example. I noticed there's bad "transition" region in this image. I replaced the threshold to `<=2` instead of `==0`, and the result was better, but it still doesn't quite look right. `rr_gray = np.where(rr_gray<=2, np.nan, rr_gray)`. I hardcoded the transition region (up to row 33) and used `rr_gray == 0`, but the result was the same. If you use more than 2, parts of the girl's hair starts to disappear. – K.Cl Apr 02 '21 at 18:04
  • When using <=2 some of the nan array elements for some reason still remain the same as I tried it on other images from my dataset, it is very strange that the code does not work as it is supposed to for these samples because the sample you created worked perfectly fine. Thank you for your assistance I will try to work with <=2 and see what I get as these images will be used to train a neural network. – Justin Apr 02 '21 at 18:14
  • This seems to be a difficult problem, I wish you luck. – K.Cl Apr 02 '21 at 18:16
  • I also tried by not converting the zero elements to nan, thus removing one line of code, and then in the `fill` function use: `if invalid is None: invalid = np.where(rr_gray == 0)` so that the zeros are replaced instead but then I got an error: `IndexError: index 112 is out of bounds for axis 1 with size 112` – Justin Apr 02 '21 at 18:18
  • I think that in this case it's better to use `if invalid is None: invalid = rr_gray <= 0` – K.Cl Apr 02 '21 at 18:33