1

I have read an image in Mat format. 

Mat image = imread("image.png", 1);

I declare a pointer to its data using 

unsigned char *ptr_source = image.data

Now, I want to access the value of R,G and B values at each pixel in a for loop. I already know the method to do it with img.at<Veb3b>(i,j) or similar things but now, I have to do it using a pointer of unsigned char type.

uchar R_value = ptr_source[ i*?? + ??? ];
uchar G_value = ptr_source[ i*?? + ??? ];
uchar B_value = ptr_source[ i*?? + ??? ];

IMPORTANT: Some people here have mentioned to use the following:

unsigned char *input = (unsigned char*)(img.data);
for(int j = 0;j < img.rows;j++){
    for(int i = 0;i < img.cols;i++){
        unsigned char b = input[img.step * j + i ] ;
        unsigned char g = input[img.step * j + i + 1];
        unsigned char r = input[img.step * j + i + 2];
    }
}

which makes sense to me as per the openCV docs but unfortunately it is not working in my case. The other method posted at SO says to use the following:

uchar b = frame.data[frame.channels()*(frame.cols*y + x) + 0];    
uchar g = frame.data[frame.channels()*(frame.cols*y + x) + 1];
uchar r = frame.data[frame.channels()*(frame.cols*y + x) + 2];

Basic Question: Though, it seems to be working but I do not understand it logically. Why do we need to multiply (frame.cols*y + x) with frame.channels() ??

Community
  • 1
  • 1
skm
  • 5,015
  • 8
  • 43
  • 104
  • 1
    Do you really have to use `data`? Anyway.. I think OpenCV store the pixels in the BGR order and the size may change according to the type of image loaded. In this [solution](http://stackoverflow.com/questions/8932893/accessing-certain-pixel-rgb-value-in-opencv) (2nd answer)... he uses `image.channels()` to know the step size. I hope his answer help you somehow. – wendelbsilva Jun 29 '15 at 13:57
  • @wendelbsilva: yes, I am looking something like that but I am not sure if that will work for me. Furthermore, I did not understand `frame.channels()*(frame.cols*y + x) + 0` for B channel and likewise. – skm Jun 29 '15 at 14:03
  • The data is stored sequentially. So the `frame.channels()*(frame.cols*y + x) + 0` means he is trying to access the pixel `(x, y)`. Since the data is sequential, the position in the array will be `y` multiplied by the width of the matrix (`frame.cols`) plus `x`. He added the `0` in the end just to help reading the algorithm. – wendelbsilva Jun 29 '15 at 14:06
  • @wendelbsilva: I am able to understand that part. My confusion is , why do we need to multiply by `frame.channels()` and how is it helping us to get R, G and B pixel values? – skm Jun 29 '15 at 14:08
  • You may have a grayscale image. In this case, the number of bytes and how they are stored will be different than when you have a RGB or RGBA image. If you know your image is RGB, you can go ahead and use the size you want. But if you dont know how many bytes it use, or if you want to use the function to make the algorithm easier to read, you can use [`.channels()`](http://docs.opencv.org/modules/core/doc/basic_structures.html#mat-channels) – wendelbsilva Jun 29 '15 at 14:11
  • what are you trying to achieve ? iterating over pixels is usally a bad idea. – berak Jun 29 '15 at 15:08
  • @berak: Actually, I am doing all the calculations at GPU. And, threadId gives me the value of `x` and `y` coordinates at GPU. Therefore, I need to know the method to access the pixel values of R, G and B components at each pixel location. – skm Jun 29 '15 at 15:11
  • Why not use GDI and GetPixel()? – Evan Carslake Jun 29 '15 at 15:12
  • @EvanCarslake: Sorry, I have just now started CUDA programming. I am not using the openCV defined CUDA methods. I am writing my own kernels in CUDA. I am not familiar with those things. May be, you can post an example about it. – skm Jun 29 '15 at 15:13
  • @skm hmm, in that case I don't think you want to use GDI. It is a Win32 library. – Evan Carslake Jun 29 '15 at 15:15

1 Answers1

2

The cv::Mat::channels() method returns the number of channels in an image.
In a 8UC3 three-channel color image, channels() returns 3, and the pixels are stored in consecutive byte-triplets: BGRBGRBGRBGRBGR....

To access pixel (x,y) given a unsigned char* ptr_source pointer, you need to calculate the pixel offset. The image width is frame.cols. Each pixel is channels() == 3 bytes, so the pixel's unsiged char* offset will be ptr_source + frame.channels()*(frame.cols*y + x). This unsigned char* would usually be the blue channel with the following 2 chars the green and red.

For example, given a 3x4 image, the pixels in memory would look like this (spaces for clarity only):

r\c  0   1   2   
 0  BGR BGR BGR
 1  BGR BGR BGR
 2  BGR>BGR<BGR
 3  BGR BGR BGR

So if you count bytes you'll see that the blue channel byte of pixel (1,2) is exactly at byte offset 3*(2*3+1) = 21

It is actually advisable to use img.step instead of the raw computation since some images have padding at the end of each pixel row so that it is not always true that img.step[0] == img.channels()*img.cols.
In this case you should use ptr_source[img.step[0]*y + img.channels()*x].

Additionally, your question assumes that the pixel depth is 8U which may not be correct for all images. If it is not, you will need to multiply everything by the depth (bytes per pixel) as well.
And this is essentially what cv::Mat:at<> does...

Adi Shavit
  • 16,743
  • 5
  • 67
  • 137
  • thanks a lot. The answer is really quite elaborated and satisfactory. I am still a little confused about the usage of `img.step`. What should I write in terms of `img.step` instead of `ptr_source + frame.channels()*(frame.cols*y + x)` ?? – skm Jun 30 '15 at 12:43
  • I updated the answer. I suggest that your do an example by hand on a piece of math paper, like I did, and you'll see that you will arrive at the same calculation yourself. – Adi Shavit Jun 30 '15 at 12:46
  • yes, I am trying to do it on a paper and I am able to understand with the first method. But the method related to `img.step` is giving incrorrect results and I can see the reason is the wrong interpretation of channel or somehthing. I used `unsigned char b_val = ptr_source[ step_size + numChannels*x + 0 ];` and so on for `r_val` and `g_val`. – skm Jun 30 '15 at 12:51
  • Actually, I think you should use `img.step[0]` as the proper width-step value. – Adi Shavit Jun 30 '15 at 12:53
  • No, it did not give the desired result. It generates a image with stripes. Shouldn't we multiply `img.step[0]` with `y` in the formula `ptr_source[img.step[0] + img.channels()*x]` which is mentioned in your answer ? – skm Jun 30 '15 at 12:57
  • Yes, of course, you need to multiply the width step by `y`. – Adi Shavit Jun 30 '15 at 12:59
  • yes, that was the problem. Anyways, now it is giving the expected results with both the methods. Thanks for an illustrative answer. – skm Jun 30 '15 at 13:01