Read cv::Mat pixel without knowing its pixel format

Question

I am aware there are several ways to read and write a pixel value of an OpenCV cv::Mat image/matrix. A common one is the .at<typename T>(int, int) method http://opencv.itseez.com/2.4/modules/core/doc/basic_structures.html#mat-at . However, this requires the typename to be known, for instance .at<double>. The same thing applies to more direct pointer access OpenCV get pixel channel value from Mat image .

How can I read a pixel value without knowing its type? For instance, it would be ok to receive a more generic CvScalar value in return. Efficiency is not an issue, as I would like to read rather small matrices.

alexisrozhkov · Answer 1 · 2015-12-14T15:06:35.867

Kind of. You can construct cv::Mat_ and provide explicit type for elements, after that you don't have to write element type each time. Quoting opencv2/core/mat.hpp

While Mat is sufficient in most cases, Mat_ can be more convenient if you use a lot of element access operations and if you know matrix type at the compilation time. Note that Mat::at(int y,int x) and Mat_::operator()(int y,int x) do absolutely the same and run at the same speed, but the latter is certainly shorter.

Mat_ and Mat are very similar. Again quote from mat.hpp:

The class Mat_<_Tp> is a thin template wrapper on top of the Mat class. It does not have any extra data fields. Nor this class nor Mat has any virtual methods. Thus, references or pointers to these two classes can be freely but carefully converted one to another.

You can use it like this

Mat_<Vec3b> dummy(3,3);
dummy(1, 2)[0] = 10;
dummy(1, 2)[1] = 20;
dummy(1, 2)[2] = 30;
cout << dummy(1, 2) << endl;

Why I said 'kind of' in the first place? Because if you want to pass this Mat_ somewhere - you have to specify it's type. Like this:

void test(Mat_<Vec3b>& arr) {
    arr(1, 2)[0] = 10;
    arr(1, 2)[1] = 20;
    arr(1, 2)[2] = 30;

    cout << arr(1, 2) << endl;
}
...
Mat_<Vec3b> dummy(3,3);
test(dummy);

Technically, you are not specifying your type during a pixel read, but actually you still have to know it and cast the Mat to the appropriate type beforehand.

I guess you can find a way around this using some low-level hacks (for example make a method that reads Mat's type, calculates element size and stride, and then accesses raw data using pointer arithmetic and casting...). But I don't know any 'clean' way to do this using OpenCV's functionality.

*if you want to pass this Mat_ somewhere - you have to specify it's type.* Not true. You can pass a `Mat_` as a `Mat` parameter (e.g. to a `void test(Mat&)` function) and it will work. — Яois, Dec 14 '15 at 15:13
What will work? Passing is surely possible, but accessing element without specifying type won't work, and that was what I meant — alexisrozhkov, Dec 14 '15 at 15:16
`void process(const cv::Mat_< float >& arr)` will accept any matrix `cv::Mat`, there is no type checking. If `arr` has `uchar` pixels, the results are wrong. — SpamBot, Dec 14 '15 at 15:54
I am not sure about this. When I try to pass `Mat` instead of `Mat_` to `void test(Mat_& arr)` I get error: `invalid initialization of reference of type ‘cv::Mat_ >&’ from expression of type ‘cv::Mat’` What compiler are you using? — alexisrozhkov, Dec 15 '15 at 11:11

Miki · Accepted Answer · 2015-12-15T11:52:01.297

If you already know the type, you can use Mat_<> type for easy access. If you don't know the type, you can:

convert the data to double, so data won't be truncated in any case
switch over the number of channels to access correctly the double matrix. Note that you can have at most of 4 channels, since Scalar has at most 4 elements.

The following code will convert only the selected element of the source matrix to a double value (with N channels).

You get a Scalar containing the value at position row, col in the source matrix.

#include <opencv2/opencv.hpp>
#include <iostream>
using namespace std;
using namespace cv;

Scalar valueAt(const Mat& src, int row, int col)
{
    Mat dst;;
    src(Rect(col, row, 1, 1)).convertTo(dst, CV_64F);

    switch (dst.channels())
    {
    case 1: return dst.at<double>(0);
    case 2: return dst.at<Vec2d>(0);
    case 3: return dst.at<Vec3d>(0);
    case 4: return dst.at<Vec4d>(0);
    }

    return Scalar();
}

int main()
{
    Mat m(3, 3, CV_32FC3); // You can use any type here
    randu(m, Scalar(0, 0, 0, 0), Scalar(256, 256, 256, 256));

    Scalar val = valueAt(m, 1, 2);
    cout << val << endl;

    return 0;
}

This looks pretty close to what I expected. However, can we avoid the `convertTo` of the full matrix? Maybe by selecting a 1x1 range on `src`? — SpamBot, Dec 15 '15 at 08:51

Read cv::Mat pixel without knowing its pixel format

2 Answers2