1

My code is all done thinking that the input Mat is in BGR format, so I am wondering if given an Image object in EmguCV, the property Mat from this object is always a BGR Mat. Otherwise, I would need to use CvtColor to get the BGR representation.

Example code:

byte[] data = GetPngPixelsArray(string); // Byte array in RGB format

Image<Rgb, byte> image = new Image<Rgb, byte>(width, height)
{
  Bytes = data
};

CvInvoke.Imshow("image mat", image.Mat);
CvInvoke.WaitKey(0);

The function I am using to get the byte array data:

internal static byte[] GetPngPixelsArray(string filename)
{
  byte[] rgbValues = null;

  using (var imageIn = Image.FromFile(filename))
  using (var bmp = new Bitmap(imageIn))
  {
    BitmapData bmpData = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height),
    ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);

    IntPtr ptr = bmpData.Scan0;

    int bytes = Math.Abs(bmpData.Stride) * bmp.Height;
    rgbValues = new byte[bytes];

    Marshal.Copy(ptr, rgbValues, 0, bytes);

    bmp.UnlockBits(bmpData);
  }

  return rgbValues;
}

In the example code above, the Imshow function is showing the image properly, and as far as I know Imshow always made a representation of the image using the BGR format.

So in fact the image.Mat is in BGR format, but I've checked the EmguCV documentation and haven't found any declaration that this is as I stated.

juan carlos
  • 184
  • 1
  • 11
  • 1
    EmguCV is a wrapper around opencv. About opencv and BGR see here: https://stackoverflow.com/questions/14556545/why-opencv-using-bgr-colour-space-instead-of-rgb. – wohlstad May 13 '22 at 11:26
  • @wohlstad yes, I know that it is a wrapper around OpenCV, thanks for the link. But I am not asking why it is using BGR format. My code is all done thinking that the input image is a BGR image, so I want to know if `image.Mat` property is always represented as a BGR image. Othewise, I would need to use `CvtColor` to turn the RGB image to BGR. Thanks for the comment. – juan carlos May 13 '22 at 11:30
  • I misunderstood then. Good question. You can consider to edit your question to emphasize this. – wohlstad May 13 '22 at 11:40
  • I checked in opencv c++, and it actually behaves differently. I created a `cv::Mat` with a rgb buffer all red, and opencv interpreted it as bgr. I.e. when I used cv::imshow I got a blue image. Are you sure your `data` is rgb ? – wohlstad May 13 '22 at 11:48
  • @wohlstad I edited my question to add the emphasis. – juan carlos May 13 '22 at 11:51
  • @wohlstad yes, I am using `Image.FromFile(string)` from `System.Drawing`, then creating a `Bitmap` from this image, and then reading the rgb values. Right now, I am thinking that when you create the `Image` object, despite the `TColor` parameter you specify, it creates a `Mat` property, and this `Mat` is always in Bgr format (you specified that the `Image` is in Rgb, so the constructor can create the `Mat` in Bgr), but I can't confirm this behaviour since EmguCV docs don't states this, and I don't find it in the source code. – juan carlos May 13 '22 at 11:55
  • So it looks like `Image` is a EmguCV wrapper around cv::Mat that handles the conversions. I actually use opencv only in c++ environment, so I'm not familiar with EmguCV. – wohlstad May 13 '22 at 11:57
  • @wohlstad I edited the question and added the way I am getting that byte array. Yes, it seems like it handles automatically the conversion to BGR given the `TColor` parameter, and then the `.Mat` property is always in a BGR format. I would like to confirm this theory, but in the docs it doesn't appear, and I haven't found yet in the source code of EmguCV. – juan carlos May 13 '22 at 12:01
  • if `Image.FromFile` is merely calling `cv::imread`, then yes, it's always BGR, unless you specifically ask for grayscale or _all channels_ of the image. – Christoph Rackwitz May 13 '22 at 13:26
  • @ChristophRackwitz but how is generated the `.Mat` property when inside the constructor of the `Image`? That's what I've been searching in the source code of EmguCV to confirm that. – juan carlos May 13 '22 at 16:24
  • @wohlstad after all, it seems like my byte array was in BGR format. I will check my data array provider to understand why it is serving the array in BGR format. Check my answer below. – juan carlos May 16 '22 at 07:04

1 Answers1

2

Just checked the behaviour of EmguCV creating an Image object, and the .Mat property has the same representation as the Image object, meaning that could be BGR or RGB.

I filled two vectors of size 750000 (500x500x3), this is what I've done:

byte[] rgbChar = new byte[750000];
byte[] bgrChar = new byte[750000];

for (int i = 0; i < rgbChar.Length; i+=3)
{
    rgbChar[i] = 255;
    rgbChar[i+1] = 0;
    rgbChar[i+2] = 0;

    bgrChar[i] = 0;
    bgrChar[i+1] = 0;
    bgrChar[i+2] = 255;
}

Image<Rgb, byte> imageRgb = new Image<Rgb, byte>(500, 500)
{
    Bytes = rgbChar
};
Image<Bgr, byte> imageBgr = new Image<Bgr, byte>(500, 500)
{
    Bytes = bgrChar
};

CvInvoke.Imshow("RGB Image", imageRgb);
CvInvoke.Imshow("RGB Image mat", imageRgb.Mat);
CvInvoke.Imshow("BGR Image", imageBgr);
CvInvoke.Imshow("BGR Image mat", imageBgr.Mat);
CvInvoke.WaitKey(0);

enter image description here

Edit: I would add some explanations, as it seems that the results are not self explaining.

I thought that the EmguCV Image class, ever was creating the Mat property as BGR. What means that?

If you check the Image creation, you need to specify the format, eg. Image<Rgb, byte>. I thought, that as you have to specify the format of the Image, that the .Mat property was a BGR representation of the RGB Image.

Basically, I thought that the behaviour was like that:

  • Image<Rgb, byte> : -> Internal conversion from RGB to BGR into the .Mat property (Wrong).
  • Image<Bgr, byte> : -> No conversion, the .Mat is BGR same as Image.

But the real behaviour, is that the .Mat property is just a Mat representation of the Image, doesn't matter if the Image is in RGB or BGR format, the .Mat property will be in the same representation.

This is the reason that in the two first cases the image shows blue (Imshow ever represents the image as BGR, if the image is in RGB it will show a blue image).

wohlstad
  • 12,661
  • 10
  • 26
  • 39
juan carlos
  • 184
  • 1
  • 11
  • Actually it looks like the data is always interpreted as BGR (you wrote _the Mat property is not always in BGR format_). In `imageRgb` the first channel is 255, but it is interpreted as blue, not red. Have I misundertood you ? – wohlstad May 16 '22 at 07:14
  • @wohlstad I've just added a explanation of the results, can you check if now is understandable? – juan carlos May 16 '22 at 09:49
  • What you added is OK, but your answer starts with "... and the Mat property is **not** always in BGR format.". Why "not" in BGR ? Isn't it actually always assumed to be BGR ? – wohlstad May 16 '22 at 09:54
  • Maybe you should change it to something like: "the internal representation of the underlying `Mat` always assumes the data is BGR". – wohlstad May 16 '22 at 09:57
  • @wohlstad the `Mat` property of the `Image` class. Eg. `Image.Mat`, I was assuming that always was BGR, but I proved that is not always BGR, is just the same representation as the `Image`. I've added another explanation on the top. – juan carlos May 16 '22 at 09:59
  • Added another clarification. I think now it is clear. – wohlstad May 16 '22 at 10:06
  • @wohlstad sorry but I don't understand what you have just edited. The `Mat` doesn't assume the first channel is the blue, this is what `Imshow` function does, but that's not what I was testing here. – juan carlos May 16 '22 at 10:09
  • I don't think imshow does that. I believe imshow is a thin wrapper around c++ cv::imshow. I think the internal `Mat` (which encapsulates the c++ cv::Mat) is simply treating the data buffer always as BGR. That is what I saw in the test I did in C++, and it fits your result. However - as I wrote to you I am not much familiar with the EmguCV wrapper, so I might be wrong in this case. Feel free to edit again and modify. – wohlstad May 16 '22 at 10:13
  • Since I am not 100% certain, I removed the sentence I previously added to your answer. – wohlstad May 16 '22 at 10:29
  • @wohlstad check this https://stackoverflow.com/questions/50963283/python-opencv-imshow-doesnt-need-convert-from-bgr-to-rgb , cv::imshow represents the `Mat` array as BGR, so I think `Mat` doesn't assume BGR format, this is what imshow does. In the case your `Mat` is in RGB format, as `cv::imshow` will assume BGR format, the image will be shown as blue. – juan carlos May 16 '22 at 10:30
  • I think it is a matter of interpretation. But anyway - what is the actual problem you are facing ? Eventually a cv::Mat holds data, and this data can be [mis]interpreted by imshow or any other mehod. If the data is BGR - opencv will work with it as is. If it is RGB (which only mean the first channel is red etc.), you can use cv::cvtColor(,...,CV_RGB2BGR) to swap the channels. BTW - note that `CV_RGB2BGR` and `CV_BGR2RGB` are defined to be the same. This is because cv::cvtColor only swaps first and last channels, it doesn't care if it's r<->b or b<->r. Is this not enough to handle your issue? – wohlstad May 16 '22 at 11:45
  • @wohlstad yes, I can face the problem with `cv::cvtColor`, but I needed to know if the `.Mat` was ever in BGR format, despite the `Image` format. Nevermind, right now my question is solved :) – juan carlos May 16 '22 at 13:05
  • Glad to hear. Sorry I couldn't assist more. – wohlstad May 16 '22 at 13:23