Detecting areas of differing colors using OpenCV?

Question

Let's say I have the following image:

I want to be able to detect each one of the white squares amongst the black background. I would also like to be able to acquire the center & size of each one.

I'm attempting to use the opencv library in C++ in order to achieve this. I'm doing this on a Raspberry Pi, with it's standard OS. I installed the latest opencv library by running the sudo apt-get install libopencv-dev command on the terminal. I don't have any prior experience with opencv or any of its libraries.

I understand that one can load an image, in the form of it's pixel data, in a variable of type cv::Mat. I understand how to access each pixel inside the cv::Mat variable, and I understand how to get the byte value for each pixel.

What I'm looking for now is a function or functions that would allow me to detect each one of the white squares, and give me their center positions on the image and their sizes.

Is what I'm trying to achieve even possible with opencv in C++? Is there a function that does what I'm looking for?

Answer 1

This is a basic operation of image processing. It is described in the OpenCV documentation.

Here's a short bit of code. Please excuse the use of Python. This looks pretty much identical in C++. All the numpy arrays are just cv::Mat objects then.

im = cv.imread("ySqre.png", cv.IMREAD_GRAYSCALE)
(nlabels, labelmap, stats, centroids) = cv.connectedComponentsWithStats(im)

And the results:

# NOTE: background is counted as a label too, with label index 0
# nlabels: 6 (5 + background)

# centroids:
# array([[320.17971, 160.72618], # this is background, ignore it
#        [ 84.5    ,  56.5    ],
#        [288.5    ,  88.5    ],
#        [437.     , 140.     ],
#        [185.     , 222.5    ],
#        [527.5    , 237.5    ]])

# for the sizes, inspect the `stats` array.
# Use cv.CC_STAT_WIDTH and CC_STAT_HEIGHT for indices into the array.
# There is also the top left corner and the area in number of pixels.

# stats:
# array([[     0,      0,    640,    320, 190359], # background, ignore
#        [    48,     39,     74,     36,   2664],
#        [   274,     62,     30,     54,   1620],
#        [   412,     81,     51,    119,   6069],
#        [   159,    195,     53,     56,   2968],
#        [   514,    218,     28,     40,   1120]], dtype=int32)

Answer 2

If you can assume the detection target area is white and its shape is (non-rotated) rectangle, simply doing labeling will achieve your detection (because of the assumption : all found white region should be rectangle).

You say you can access pixel data, so, you can implement simple labeling process for this purpose.

Very simple sample code:

#include <iostream>
#include <vector>

struct HorizontalLineSeg
{
    int left, right, y;
    HorizontalLineSeg( int left=0, int right=0, int y=0 )
        : left(left), right(right), y(y)
    {}
};

struct RectReg
{
    HorizontalLineSeg Top, Bottom;
    RectReg( const HorizontalLineSeg &Top ) : Top(Top),Bottom(Top) {}
    bool ConnectToBottom( const HorizontalLineSeg &Seg )
    {
        if( Bottom.y+1 != Seg.y )return false;
        if( Seg.right < Bottom.left  ||  Bottom.right < Seg.left )return false;
        Bottom = Seg;
        return true;
    }
};

void Update( std::vector<RectReg> &Regs, const HorizontalLineSeg &FoundSeg )
{
    for( auto &Reg : Regs )
    {
        if( Reg.ConnectToBottom( FoundSeg ) )return;
    }
    Regs.emplace_back( FoundSeg);
}

int main()
{
    //- Load Image as GrayScale
    cv::Mat Img = cv::imread( "WhiteRects.png", cv::IMREAD_GRAYSCALE );
    if( Img.empty() ){  std::cout << "imread() failed" << std::endl;    return 0;   }
    //- Find white regions
    std::vector<RectReg> Regs;
    {
        const unsigned char Thresh = 128;
        for( int y=0; y<Img.rows; ++y )
        {
            const unsigned char *p = Img.ptr<unsigned char>( y );
            int FoundLeft = -1;
            for( int x=0; x<Img.cols; ++x, ++p )
            {
                if( *p >= Thresh )
                {
                    if( FoundLeft<0 )FoundLeft = x;
                }
                else if( FoundLeft >= 0 )
                {
                    Update( Regs, HorizontalLineSeg( FoundLeft, x-1, y ) );
                    FoundLeft = -1;
                }
            }
        }
    }
    //- Visualize result
    cv::Mat ShowImg = Img * 0.35;
    if( !Regs.empty() )
    {
        std::vector< std::vector<cv::Point> > Pts;
        Pts.reserve( Regs.size() );
        for( const auto &Reg : Regs )
        {
            Pts.push_back(
                {
                    { Reg.Top.left, Reg.Top.y },
                    { Reg.Top.right, Reg.Top.y },
                    { Reg.Bottom.right, Reg.Bottom.y },
                    { Reg.Bottom.left, Reg.Bottom.y }
                }
            );
        }
        cv::polylines( ShowImg, Pts, true, cv::Scalar(255) );
    }
    std::cout << Regs.size() << " regs found" << std::endl;
    cv::imshow( "Result", ShowImg );
    cv::waitKey();
    return 0;
}

---

Added code that uses OpenCV functions because it seems to be criticized for not using OpenCV functions.

int main()
{
    //- Load Image as GrayScale
    cv::Mat Img = cv::imread( "WhiteRects.png", cv::IMREAD_GRAYSCALE );
    if( Img.empty() ){  std::cout << "imread() failed" << std::endl;    return 0;   }
    //- Main process with cv::connectedComponentsWithStats()
    cv::Mat Stats;
    int N = 0;
    {
        //- Pre-Binalize with OpenCV's function.
        //  Purpose of this is just to set the background value to 0.
        //  So, if you know the backgroud value is already 0, you can omit this step.
        cv::Mat BinImg;
        cv::threshold( Img, BinImg, 128, 255, cv::THRESH_BINARY );
        //- Use connectedComponentsWithStats()
        cv::Mat Labels;  //Only needed to use the function.
        cv::Mat Centroids;  //Only needed to use the function. Or you can use this if you want.
        N = cv::connectedComponentsWithStats( BinImg, Labels, Stats, Centroids );
    }
    //- Visualize result
    cv::Mat ShowImg = Img * 0.35;
    for( int i=1; i<N; ++i )    //be careful not to include 0
    {
        const int *pS = Stats.ptr<int>( i );
        cv::rectangle(
            ShowImg,
            cv::Rect(
                cv::Point{ pS[ cv::CC_STAT_LEFT ],  pS[ cv::CC_STAT_TOP ] },
                cv::Size{ pS[ cv::CC_STAT_WIDTH ],  pS[ cv::CC_STAT_HEIGHT ] }
            ),
            cv::Scalar(255)
        );
    }
    std::cout << N-1 << " regs found" << std::endl; //not N
    cv::imshow( "Result", ShowImg );
    cv::waitKey();
    return 0;
}