0

Goal: I wish to merge thermal and RGB images into new 4-channel images for DNNs (perhaps pre-trained ones like YOLO) to improve detection and tracking—akin to how the computer vision of autonomous vehicles fuses multiple input sources before applying DNNs.

Question: My problem is that my new images are washed-out. The thermal channel is the alpha channel. Is there a way to use OpenCV (and Python) to change the opacity of an image's alpha channel?

My test color and thermal images are the following:

enter image description here

enter image description here

My code is the following:

import cv2
import matplotlib.pyplot as plt
import matplotlib.image as mpimg

bgr_img = mpimg.imread("color.png")
th_img = mpimg.imread("thermal.png")

b, g, r = cv2.split(bgr_img)

bgrth_img = cv2.merge([b, g, r, th_img])

Which returns the following image:

enter image description here

How Do I use OpenCV and Python to change the transparency or opacity of the thermal channel as the alpha channel? Also, do you reckon that merging thermal and RBG images will increase detection and tracking accuracy?

Ashton T. Sperry
  • 105
  • 1
  • 7
  • Well, your 4th channel is not alpha transparency, so if you feed it to functions that expect it to be alpha transparency, you won't get a meaningful result. You will most likely have to do some preprocessing before you use a standard drawing function to show the image. That's something you haven't really defined -- what effect should the IR channel have on the RGB image you're trying to combine it with? Some sort of colour-mapped version of the IR channel, alpha blended with the RGB? Something else? Can you concoct some illustration of the desired output? – Dan Mašek Nov 02 '22 at 23:29
  • 1
    Perhaps PCA reduction from 4 channels to 3 ???? Sorry, I do not have experience with DNNs. – fmw42 Nov 02 '22 at 23:32
  • @fmw42 I have a feeling it doesn't matter to the network, it's just a question of how to visualize it. – Dan Mašek Nov 02 '22 at 23:47
  • 1
    If you want to increase contrast, you can use either skimage.exposure.rescale_intensity or cv2.normalize. I prefer the former. See https://stackoverflow.com/questions/42257173/contrast-stretching-in-python-opencv/59515897#59515897 and https://stackoverflow.com/questions/73971444/binarize-low-contrast-images/73979467#73979467 – fmw42 Nov 03 '22 at 04:27
  • 1
    That is an excellent question, @DanMašek. I desire merged images to isolate those objects with thermal signatures and fade the rest into the background. The faded image above does perform the function somewhat, but I'd like the fire extinguisher and person to be more prominent with the light background. Does that make sense? The object detection and tracking I'm interested in will have a much easier time with merged images, as I've described. – Ashton T. Sperry Nov 03 '22 at 08:14

0 Answers0