Estimate white background

Question

I have image with white uneven background (due to lighting). I'm trying to estimate background color and transform image into image with true white background. For this I estimated white color for each 15x15 pixels block based on its luminosity. So I've got the following map (on the right): Now I want to interpolate color so it will be more smooth transition from 15x15 block to neighboring block, plus I want it to eliminate outliers (pink dots on left hand side). Could anyone suggest good technique/algorithm for this? (Ideally within OpenCV library, but not necessary)

Answer 1

Starting from this image:

You could find the text on the whiteboard as the parts of your images that have a high gradient, and apply a little dilation to deal with thick parts of the text. You'll get a mask that separates background from foreground pretty well:

Background:

Foreground:

You can then apply inpainting using the computed mask on the original image (you need OpenCV contrib module photo):

---

Just to show that this works independently of the text color, I tried on a different image:

Resulting in:

---

Code:

#include <opencv2/opencv.hpp>
#include <opencv2/photo.hpp>
using namespace cv;

void findText(const Mat3b& src, Mat1b& mask)
{
    // Convert to grayscale
    Mat1b gray;
    cvtColor(src, gray, COLOR_BGR2GRAY);

    // Compute gradient magnitude
    Mat1f dx, dy, mag;
    Sobel(gray, dx, CV_32F, 1, 0);
    Sobel(gray, dy, CV_32F, 0, 1);
    magnitude(dx, dy, mag);

    // Remove low magnitude, keep only text
    mask = mag > 10;

    // Apply a dilation to deal with thick text
    Mat1b K = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));
    dilate(mask, mask, K);
}

int main(int argc, const char * argv[])
{
    Mat3b img = imread("path_to_image");

    // Segment white
    Mat1b mask;
    findText(img, mask);

    // Show intermediate images
    Mat3b background = img.clone();
    background.setTo(0, mask);

    Mat3b foreground = img.clone();
    foreground.setTo(0, ~mask);

    // Apply inpainting
    Mat3b inpainted;
    inpaint(img, mask, inpainted, 21, CV_INPAINT_TELEA);

    imshow("Original", img);
    imshow("Foreground", foreground);
    imshow("Background", background);
    imshow("Inpainted", inpainted);
    waitKey();

    return 0;
}