1

I am working with a custom USB camera which sends raw Bayer BGGR 16 bit/pixel images. I have a C++ project which is able to show the camera stream in grayscale, but when I try to use any of the OpenCV's built in color conversion function I always got misscolored images. I checked the camera picture and also with the sensor test pattern feature. I tried demosaicing the picture manually, only got greenish distorted pictures. I tried to learn more about the bayer/raw images, but now I am a bit stuck and would be glad for any kind of help or hint. It is also looks strange for me, when I check the type and channels of the input videocapture frame It has 8bit 3 channel, even if try to turn of the RGB conversion. I attached the actuall outputs.

Grayscale test pattern:

enter image description here

COLOR_BayerRG2RGB:

enter image description here

Manual demosaic:

enter image description here

Gray camera picture:

enter image description here

COLOR_BayerRG2RGB:

enter image description here

Manual demosaic:

enter image description here

LB91
  • 21
  • 6
  • With some effort I managed to get the following [image](https://i.stack.imgur.com/G928U.png). I suspect your optics is missing an optical element called "IR Cut-Off Filter", but there are many other possibilities. Your "Manual Demosaic" is not implemented correctly. – Rotem Mar 27 '22 at 20:58
  • @rotem Your result really impressive. Yes I realized my demosaicing is a total misconception. I read more about the topic and realized I should rely on the built in COLOR_BayerGB2BGR function and try to correct the colors by white balancing and/or gamma mapping. I currently working on that. Can I ask you about the tools/efforts you made to have that result image? – LB91 Mar 28 '22 at 07:18
  • No, it's just going to distruct you... I think something is wrong with your setup. You also have to learn to post an unprocessed raw image, and learn to post the relevant stuff. Do you know if you have an IR cutoff filter? – Rotem Mar 28 '22 at 08:26
  • @rotem I tested the sensor with a different frontend and did not experienced that kind of yellow overlay thats why I think the sensor works properly. Uncompress raw image post mean I have to saveimage as .tiff or something similar? – LB91 Mar 28 '22 at 12:59
  • 16 bits Grayscale Tiff or raw binary. But you can't post in Stack Overflow. Find other site to share it. – Rotem Mar 28 '22 at 14:28
  • What do you mean "frontend"? Different software? – Rotem Mar 28 '22 at 14:32
  • @rotem In this case the frontend means the interface IC which is communicate with the image sensor(via an i2c interface). This frontend reads the the analog video signal and creates the parallel digital video signals. – LB91 Mar 28 '22 at 18:09
  • That means that the Camera has the functionality needed for processing the video (apply Demosaicing, White Balance, Gamma...). Can't you configure the camera to transmit processed video instead of raw Bayer? In case you have to capture raw Bayer, maybe you should configure the camera to turn off some of the internal processing that is on by default (disabling all processing like automatic brightness, contrast, White Balance...) – Rotem Mar 28 '22 at 19:36
  • @rotem based on the datasheet the sensor sending 10 bit raw rgb pixels. For interfacing the camera I have created a firmware which makes a UVC Usb device. But the UVC class not support the raw format so I wrote into the descriptors the RGB555 format and in the host side I handle the bit orde and create the grayscale image. After that I using the fact that the marketing datasheet says this sensor has a bggr bayer. But yes via the mcu I can tune the gamma white balance properties of the sensor. – LB91 Mar 30 '22 at 06:01

1 Answers1

0

Try cv::COLOR_BayerGB2BGR instead of cv::COLOR_BayerRG2RGB.

OpenCV supports 4 bayer patterns:

COLOR_BayerBG2BGR
COLOR_BayerGB2BGR
COLOR_BayerRG2BGR
COLOR_BayerGR2BGR

So either your pattern (BGGR) correlates to cv::COLOR_BayerGB2BGR, or it could be that you actually have one of the other patterns.

Anyway a quick test will determine it.

wohlstad
  • 12,661
  • 10
  • 26
  • 39
  • Thanks for your suggestion, I tried them and BG2BGR is the closest to the reality, but that still contains a yellowish overlay. – LB91 Mar 26 '22 at 14:16