OpenCV ROI Out-of-bounds: Fill with black?

Question

I'm using OpenCV to get a small rectangular ROI from a large image and save the ROI to a file. Sometimes, the ROI goes out of bounds of the image. I need a way to make the resulting Mat show the portion of the large image that is within bounds, and show black for the rest.

To help explain, image that you have an image that is a map of an area. I know where a person is on the map, and want to take a 500x500 pixel section of the map with their location in the center. But when the user gets to the edge of the map, part of this 500x500 section will need to be "off the map". So I would like it to fill it in with black.

Preferably, OpenCV would be able to gracefully handle out-of-bounds ROIs (e.g. negative top left corner values) by filling in with black (like it does when you rotate an image with warpAffine) but that doesn't seem to be the case. Any suggestions for how to accomplish this goal?

Answer 1

All other answers seem a little bit too complicated to me. Simply:

// Create rects representing the image and the ROI
auto image_rect = cv::Rect({}, image.size());
auto roi = cv::Rect(-50, 50, 200, 100)

// Find intersection, i.e. valid crop region
auto intersection = image_rect & roi;

// Move intersection to the result coordinate space
auto inter_roi = intersection - roi.tl();

// Create black image and copy intersection
cv::Mat crop = cv::Mat::zeros(roi.size(), image.type());
image(intersection).copyTo(crop(inter_roi));

Image for reference:

Answer 2

I found that the best way to do this was to get the section of the ROI that was within bounds, then calculate how much on each side (top/bottom/left/right) of the ROI was out of bounds, then use the copyMakeBorder function to pad that much black border around each side. It worked out very well. It looks something like this now:

Mat getPaddedROI(const Mat &input, int top_left_x, int top_left_y, int width, int height, Scalar paddingColor) {
    int bottom_right_x = top_left_x + width;
    int bottom_right_y = top_left_y + height;

    Mat output;
    if (top_left_x < 0 || top_left_y < 0 || bottom_right_x > input.cols || bottom_right_y > input.rows) {
        // border padding will be required
        int border_left = 0, border_right = 0, border_top = 0, border_bottom = 0;

        if (top_left_x < 0) {
            width = width + top_left_x;
            border_left = -1 * top_left_x;
            top_left_x = 0;
        }
        if (top_left_y < 0) {
            height = height + top_left_y;
            border_top = -1 * top_left_y;
            top_left_y = 0;
        }
        if (bottom_right_x > input.cols) {
            width = width - (bottom_right_x - input.cols);
            border_right = bottom_right_x - input.cols;
        }
        if (bottom_right_y > input.rows) {
            height = height - (bottom_right_y - input.rows);
            border_bottom = bottom_right_y - input.rows;
        }

        Rect R(top_left_x, top_left_y, width, height);
        copyMakeBorder(input(R), output, border_top, border_bottom, border_left, border_right, BORDER_CONSTANT, paddingColor);
    }
    else {
        // no border padding required
        Rect R(top_left_x, top_left_y, width, height);
        output = input(R);
    }
    return output;
}

And you can easily make the padding whatever colour you like, which is nice.

Answer 3

I am not sure about a function doing that for you, but it's quite simple to do this by your self - I made a similar function to constrain a ROI. You just need to compare the topLeft position of the ROI with the image borders.

e.g. check

if(roi.bottomRight.x > image.bottomRight.x) 
    constrain.x = image.bottomRight.x - roi.topLeft.x

This is your access point in your ROI matrix, containing the tracked image. You would need to do this for every border. Now you can just access the ROI matrix inside these borders and set the pixel values to (0,0,0).

Another workaround would be possible to create a second rectangle(in size of the image) and use the rectangle operators for intersection(rect = rect1 & rect2). Then you've got the constraints right away by subtracting the width and height of the tracked rectangle and the intersection rectangle and you are able to perform the same matrix access like I mentioned above. If it's a possible alternative solution - you can just use the intersection rectangle without the black regions - then you just have to copy the values inside the range of the intersection rectangle's size.