11

I am writing a code to change the color of hair in the facial picture of a person. Doing this I made a model and was able to get a mask of the parts of the hair. But now I am stuck at a problem how to change the color of it.

Below is the output mask and input image passed.

enter image description here enter image description here

Can you suggest me the method that could be used to change the color of the hair into different colors?

yatu
  • 86,083
  • 12
  • 84
  • 139
Pratham Jindal
  • 123
  • 1
  • 1
  • 6

1 Answers1

28

Since they both have the same shape, you can mask the image of the face using mask image. We first need to perform binary thresholding on it, so it can be used as a b&w mask. Then we can perform boolean indexing based on whether a value is 0 or 255, and assign a new color, such as green?

import cv2
mask = cv2.imread('eBB2Q.jpg')
face = cv2.imread('luraB.jpg')

_, mask = cv2.threshold(mask, thresh=180, maxval=255, type=cv2.THRESH_BINARY)
# copy where we'll assign the new values
green_hair = np.copy(face)
# boolean indexing and assignment based on mask
green_hair[(mask==255).all(-1)] = [0,255,0]

fig, ax = plt.subplots(1,2,figsize=(12,6))
ax[0].imshow(cv2.cvtColor(face, cv2.COLOR_BGR2RGB))
ax[1].imshow(cv2.cvtColor(green_hair, cv2.COLOR_BGR2RGB))

enter image description here

Now we can combine the new image with the original using cv2.addWeighted, which will return the weighted sum of both images, hence we'll only see a difference on the masked region:

green_hair_w = cv2.addWeighted(green_hair, 0.3, face, 0.7, 0, green_hair)

fig, ax = plt.subplots(1,2,figsize=(12,6))
ax[0].imshow(cv2.cvtColor(face, cv2.COLOR_BGR2RGB))
ax[1].imshow(cv2.cvtColor(green_hair_w, cv2.COLOR_BGR2RGB))

enter image description here

Note that you can set the weights in the weighted sum via the alpha and beta parameters, depending on how much you want the new colour to predominate. Note that, as mentioned earlier the new image will be obtained from the weighted sum dst = src1*alpha + src2*beta + gamma. Let's try with another colour and setting the weights as a convex combination with alpha values ranging from say 0.5 and 0.9:

green_hair = np.copy(face)
# boolean indexing and assignment based on mask
green_hair[(mask==255).all(-1)] = [0,0,255]
fig, axes = plt.subplots(2,2,figsize=(8,8))
for ax, alpha in zip(axes.flatten(), np.linspace(.6, .95, 4)):
    green_hair_w = cv2.addWeighted(green_hair, 1-alpha, face, alpha, 0, green_hair_w)
    ax.imshow(cv2.cvtColor(green_hair_w, cv2.COLOR_BGR2RGB))
    ax.axis('off')

enter image description here

yatu
  • 86,083
  • 12
  • 84
  • 139
  • 7
    Mmm... stylish! – Mark Setchell Jul 14 '20 at 09:46
  • I believe the OP wanted to change the color of her hair, not remove it and put a green background in its place. You should be able to color the hair green by combining the color with her hair. You could use cv2.addWeighted to accomplish that. – Tyson Jul 14 '20 at 10:04
  • 2
    Yes, thought about that, updated. Though the limitations are clear given the current mask @Tyson – yatu Jul 14 '20 at 10:06
  • 2
    Wow that looks good! Much better than I was expecting. Another idea could be to convert her hair to the HSV color space, change the hue a bit, and convert it back to BGR before masking it in. That might improve the dark areas of her hair from becoming too bright of a green. – Tyson Jul 14 '20 at 10:15
  • 2
    That sounds like a good idea. Will give it a try when I have a little more time :) thx again @Tyson – yatu Jul 14 '20 at 10:17
  • Thank you so much for the answer. That was a great approach. Also, I wanted to ask why did you use ".all(-1)" in your code while masking, my code didn't work with that I had to remove that to get the same results. – Pratham Jindal Jul 14 '20 at 18:25
  • Because I'm checking that all elements along the last axis (channel) are `255`. Since I've made the mask `0` or `255` valued, this sets to true those rows that are all `255`. And I index the other image on these rows, and set them to `[0,255,0]` @PrathamJindal – yatu Jul 14 '20 at 18:51
  • btw, is your approach to get the mask for the hair very *long*? If not, would you mind sharing it? @PrathamJindal – yatu Jul 14 '20 at 19:21
  • I have used a CNN trained model for it trained on the CelebAMask dataset. @yatu – Pratham Jindal Jul 15 '20 at 06:18