0

So I've had a go at different way of manipulating the current image. I have the following image and subsequent mask for the image:

enter image description here

enter image description here

Recently I have been pointed to a method such as contrast enhancement of an image. I have looked potential ways to do it such as hsv splitting and appling the mask but have not been getting the results I'm looking for. Is there a way increase the contrast of the image so the areas of the image which have the saliency are brighter and the areas of low saliency aren't so bright. For example the image below, I want to try and get the same kind of result. I've looked at the following Automatic contrast and brightness adjustment of a color photo of a sheet of paper with OpenCV but haven't had much luck in regards to anything.

enter image description here

Nhyi
  • 373
  • 1
  • 12
  • I pointed you to hard light composition in two other of your posts. Does that not do what you want here. – fmw42 Jul 30 '21 at 20:06
  • Are you dealing with color images or grayscale? Which do you want to adjust? – fmw42 Jul 30 '21 at 20:08
  • See https://stackoverflow.com/questions/68544861/adding-images-together-with-a-mask/68552365#68552365 – fmw42 Jul 30 '21 at 20:14
  • Oh yeah I tried that and found that even though the results are good. It's not keeping the colour that I wanted. I then tried HSV but the saturation didn't achieve what I wanted. I'm dealing with colour images, the grayscale image is areas that I want to emphasize on the painting. I want to be able to increase the contrast of the colours in the painting in those areas. – Nhyi Jul 30 '21 at 20:15
  • That is just what I gave you. I do not understand what it is you want if those two techniques do not do that for you. Can you show an before and after of what you want? – fmw42 Jul 30 '21 at 20:16
  • That does a lot of what I want, but I want to minimize how white the regions of high salience are and instead just increase the intensity of those colours already in or increase contraste between those and surrounding areas while not making the image too different from the original style. – Nhyi Jul 30 '21 at 20:17
  • Then adjust the saliency map so that it has less contrast and is still average of mid gray. – fmw42 Jul 30 '21 at 20:18
  • I'll give this a go. This project is meant to work for multiple different images, so I'm trying to not have the need to tune each map individually which is what is difficult at the moment since I'm trying to functionalize the code you have given me. – Nhyi Jul 30 '21 at 20:22
  • Each saliency map will be different and will need adjusting. You might see if something like CLAHE might modified it more universally for any given saliency map. – fmw42 Jul 30 '21 at 20:38

1 Answers1

1

Here is the hard light composition in Python/OpenCV using an intensity modified saliency map. You can adjust the arguments in the rescale_intensity to adjust as desired.

Image:

enter image description here

Saliency:

enter image description here

import cv2
import numpy as np
import skimage.exposure

# read image 1
img12 = cv2.imread('img12.png')
hh, ww = img12.shape[:2]

# read saliency mask as grayscale and resize to same size as img1
mask = cv2.imread('hard_light_mask.png')
mask = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
mask = cv2.resize(mask, (ww,hh))
mask = cv2.merge([mask,mask,mask])

# adjust mask contrast and brightness
mask = skimage.exposure.rescale_intensity(mask, in_range=(0,255), out_range=(92,192)).astype(np.uint8)
print(mask.dtype)

# threshold mask at mid gray and convert to 3 channels
thresh = cv2.threshold(mask, 128, 255, cv2.THRESH_BINARY)[1]

# do hard light composite of img12 and mask
# see CSS specs at https://www.w3.org/TR/compositing-1/#blendinghardlight
img12f = img12.astype(np.uint8)/255
maskf =  mask.astype(np.uint8)/255
threshf =  thresh.astype(np.uint8)/255
threshf_inv = 1 - threshf
low = 2.0 * img12f * maskf
high = 1 - 2.0 * (1-img12f) * (1-maskf)
result = ( 255 * (low * threshf_inv + high * threshf) ).clip(0, 255).astype(np.uint8)

# save results
cv2.imwrite('img12_reduced_hardlight.png', result)

# show results
cv2.imshow('img12', img12)
cv2.imshow('mask', mask)
cv2.imshow('thresh', thresh)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()

Result:

enter image description here

fmw42
  • 46,825
  • 10
  • 62
  • 80
  • OMG THIS IS IT. Thank you so much, this is exactly what I was trying to go for, it highlights the regions that are wanted without taking away from the style too much. – Nhyi Jul 30 '21 at 20:49
  • In regards to CLAHE it's definitely something that I need to look into and my professor has recommended that I look into some of these techniques but I also need to make sure that I'm able to do it so that multiple images end up working for this. – Nhyi Jul 30 '21 at 20:56
  • However, I notice that on the resultant image, it seems that the map is slightly lower than where they should be if that makes sense? The region for the eyes are lower than the actual eyes. Is that due to a resizing issue? – Nhyi Jul 30 '21 at 20:58
  • Yes, that would be a resize issue. How do you know that they should align when simply resized? – fmw42 Jul 30 '21 at 21:15
  • If you need it to be a little higher, you can trim off one row at the top of the saliency image before resizing and see if that helps. Otherwise, you may have to pick control points. – fmw42 Jul 30 '21 at 21:16
  • so I just window copy and pasted them into the stack overflow question when in reality they should be the same size as the salience map is used on the original picture and then reused again later on so it should be the same size regardless as long as dimensions are the same but with my neural network that does all the simulations to create this style transfer, I have different dimensions on the images. – Nhyi Jul 30 '21 at 21:20
  • Sorry, I do not understand. Do you have a correct saliency image for that color image? If so, use it. Otherwise, you will have to find a way to align them properly via control points and affine or perspective warping. – fmw42 Jul 30 '21 at 21:22
  • oh that is what I mean sorry for rambling. I have the original saliency map for the colour image that I can use instead. Thank you so much for your help voer the past couple of days it has been quite a lot to take in and I wasn't too specific for certain parts of it. I'll have a go at this tomorrow and then see what success I have! – Nhyi Jul 30 '21 at 21:24
  • You can adjust the contrast by adjusting the out_range=(92,192) in the rescale_intensity. My guide was to start by make the range smaller but centered at 128. The smaller the range the less contrast change. The smaller the mid-value the darker it will be. So I tried once at 64,192 so that the center was 128. But it was a bit too dark. So I raised the lower value to 92. Unfortunately, this approach will likely need adjusting for different saliency maps unless your process makes them about the same. So that is why I suggested you look into CLAHE. I do not know if that will help do it automatic. – fmw42 Jul 30 '21 at 21:31
  • That's fair enough. I'll have a look into it and see if I can get maps to be the same but not sure how successful it will be. – Nhyi Jul 30 '21 at 21:43
  • Apart from CLAHE, you might find the current method performs better if you change in_range=(0,255) to in_range='image' in rescale_intensity. But you may have to modify the out_range values from what I used a little to adjust for the change of in_range. Once you do that, it may perform better. – fmw42 Jul 30 '21 at 21:48
  • I'll give this a try tomorrow and let you know how it goes. It seems pretty good and I will need to do some more research on CLAHE but it seems good so far. – Nhyi Jul 30 '21 at 22:35
  • For CLAHE, see for example, https://www.pyimagesearch.com/2021/02/01/opencv-histogram-equalization-and-adaptive-histogram-equalization-clahe/ – fmw42 Jul 30 '21 at 23:32