0

I have a series of raw HD images at a resolution of 1920x1080px. These series of images may or may not contain one or more colored rectangles. Using some C++ code, I'd like to process the decoded RGBA images to locate these rectangles (if they exist).

Instead of writing something from first principles, I was wondering if there was an existing framework I could use to detect/report the location (x/y coordinates) of the shapes within a given image. Is a library like OpenCv capable?

Criteria are as follows:

  • Framework that will work with C/C++
  • Performance is key - doing this real-time would be advantageous (say @ 50fps)
  • The color of the rectangles would be known before the run. I have no experience with computer vision, but doing a web search, I see lots of posts requiring training sets - I don't want that.
  • Detecting only rectangles for now is fine, but it would be great to be able to expand this to say circles but I want to start simple.
  • The images to-be-searched may contain lots of noise, but the colour of the shape will be unique.

Attached is a simplified example image (without much noise) containing a green rectangle; I'd like detect the x/y coordinate of the green rectangle.

Any tips/suggestions would be much appreciated.

Example of an image to be searched

ZeroDefect
  • 663
  • 1
  • 8
  • 27
  • I recommend posting to [softwarerecs.se], since asking for recommendations is off-topic for StackOverflow. – Thomas Matthews Dec 01 '20 at 00:08
  • I do recommend writing it from scratch. Since the color is known, you can easily obtain the mask, do some [morphological operations](https://www.mathworks.com/help/images/morphological-dilation-and-erosion.html) to remove noise and get the bounding box. You can get say 200 FPS. No training required, this is not a machine learning task. – Burak Dec 01 '20 at 00:33
  • I do agree with Burak, Since the colour seems to be unique you could go for a colour "filter" here : https://docs.opencv.org/3.4/da/d97/tutorial_threshold_inRange.html convert it to black and white (speedup) only marking your region of interest and apply edge detection with optional bounding rectangle for your coordinates of your marked polygon – t2solve Dec 01 '20 at 17:40

1 Answers1

0

I can at least give u a hint

#include <iostream>
#include <vector>
#include <Windows.h>

int main()
{
    int x = 0;
    int y = 0;
    int w = 100;
    int h = 100;

    int screen_w = GetSystemMetrics(SM_CXFULLSCREEN);
    int screen_h = GetSystemMetrics(SM_CYFULLSCREEN);

    HDC hdc = GetDC(HWND_DESKTOP);
    HBITMAP hbitmap = CreateCompatibleBitmap(hdc, screen_w, screen_h);
    HDC memdc = CreateCompatibleDC(hdc);
    HGDIOBJ oldbmp = SelectObject(memdc, hbitmap);
    BitBlt(memdc, 0, 0, w, h, hdc, x, y, CAPTUREBLT | SRCCOPY);
    SelectObject(memdc, oldbmp);

    BITMAPINFOHEADER infohdr = { sizeof(infohdr), w, h, 1, 32 };
    int size = w * h * 4;
    std::vector<BYTE> bits(size);
    int res = GetDIBits(hdc, hbitmap, 0, h, &bits[0],
        (BITMAPINFO*)&infohdr, DIB_RGB_COLORS);
    if (res != h)
    {
        std::cout << "error\n";
        return 0;
    }

    BYTE* ptr = bits.data();
    //for(y = 0; y < h; y++)
    for (y = h - 1; y >= 0; y--) //bitmaps bits start from bottom, not top
    {
        for (x = 0; x < w; x++)
        {
            BYTE blu = ptr[0];
            BYTE grn = ptr[1];
            BYTE red = ptr[2];
            ptr += 4;
        }
    }

    SelectObject(memdc, oldbmp);
    DeleteObject(hbitmap);
    ReleaseDC(HWND_DESKTOP, hdc);
    DeleteDC(memdc);

    return 0;
}

This is a Screenregion Pixelscan Code The following link is a repos to a cursor position color detection project I've implemented with it. When u understand the code and your problem you could be able to take it on your own :) https://github.com/Rick-laboratory/Windows-CursorPosition-ColorDetection/blob/master/main.cpp

  • Take a look at my answer here: https://stackoverflow.com/questions/7263621/how-to-find-corners-on-a-image-using-opencv/7263794#7263794 The referenced OpenCV tutorial almost solves the problem out of the box. – Throwback1986 Dec 01 '20 at 01:49
  • Thank you for your suggestion, I learned something. –  Dec 01 '20 at 02:57
  • @Throwback1986 what is the performance like? – ZeroDefect Dec 01 '20 at 09:40
  • Scanning is as performant as ur system can get because this approach takes one shot only, but when u want to take a series of shots to analyze. I would suggest working with DirectX on windows. I did not use OpenCV yet. –  Dec 01 '20 at 23:17