detect areas with inverted background/foreground

Question

How to detect areas where the background and foreground is inverted (where the text is black with a white background.. I want to invert these areas so all text is white with a black background

There are 3 areas where text is black with a white background

Just get the maximum value of the image and do, image = (maxValue - image). Have a look at this: https://stackoverflow.com/questions/19580102/inverting-image-in-python-with-opencv — Rick M., Mar 04 '19 at 14:52
I don't want to invert the whole image.. only the areas where the text is black on a white background — clarkk, Mar 04 '19 at 14:56
you have to search for the greater white rectangles containing some black pixels, and to reverse them. Probably you have to put a minimal rectangle size to not confuse a letter/small picture with a white rectangle area. It seems difficult to have a font recognition. The white lines have no black pixels inside and will not be impacted. — bruno, Mar 04 '19 at 14:59

Aziuth · Answer 1 · 2019-03-05T13:01:01.780

My approach would be to go like that:

There is some structure which stores which pixels already have been handled.

Write a routine bool belongs_to_black_text(const PixelCoordinates& coords); which will be applied on every pixel in the image. Replace the argument accordingly to however you handle coordinates.

Check if the pixel was already handled, if so, return (we need this to be time efficient)
Check whether the pixel is black, else return.
Find the connected component of black pixels. Mark all its pixels as handled.
Find the union of the connected components of all white pixels adjacent to the black component. Mark all its pixels as handled.
Check whether the frustrum of the white component is bigger than that of the black connected component. If it is, it indicates that it surrounds the black component, thus the black component is text.
Check whether the frustrum of the white component has a certain minimal size, in order to rule out that it is a white letter with a black hole.
Act accordingly (invert those).

As stept 6 inverts also the white component, we need to work on two images, the original and the result, we need the original image to retain the white component.

An improvement would be to mark previously white components surrounding text somehow so that we do not have to find the component anew for each new letter.

Edit: Changed the single white connected component to the union of the connected components of all white pixels adjacent to the black component. Else, the white component could have been something like the eye within some letter.

Note: Maybe a better approach is to start with white pixels, create the connected component, then find out every connected component within the white component and invert the whole thing.

Edit: added another step, which is now step 6.

Pseudocode:

Given:

Pixel: both coordinate and content
PixelList: stores a list of Pixel
Greyscale: a pixel can be evaluated to it's greyscale value within 0..1
is black: true iff Greyscale < 0.5
is white: not is black
Image: 2D-array of Pixel, accessed via image(pixel)
minimal_size: to be too big to be assumed to be a letter

Supporting algorithms:

connected_component(pixel, condition): creates the connected component wrt the condition, using something like BFS,
example: "find me all black pixels connected to this pixel over other black pixels"
adjacent(pixel_list, condition): all pixels that are adjacent to any pixel in the given list that fulfill the condition
Frustrum(pixel_list): finds the minimal and maximal coordinates of the list, both in x- and y-direction

Algorithm:

instantiate 2D-array of bool that is named handled, same size as image, default value false
instantiate a copy of image named result

for each pixel in image:
    if handled(pixel) continue
    if pixel is white continue

    PixelList black_component = connected_component(pixel, is black)
    PixelList white_seed = adjacent(black_component)
    PixelList white_component
    for each white_pixel in white seed:
        merge connected_component(white_pixel, is white) into white_component

    handled(white_component) = true
    handled(black_component) = true

    if Frustrum(white_component) < minimal_size continue
    if Frustrum(black_component) is within Frustrum(white_component)
        invert white_component and black_component in result

@clarkk You mean implementing the whole algorithm? No, that is your job. You decide on some criterium of how to detect black text and then you go at it. By the way, in the case of your example image, all areas in question are rectangular, you could simply search for white rectangles if all images you might get have that property. — Aziuth, Mar 04 '19 at 15:32
@clarkk My algorithm is wrong, wait for an update. Edit: fixed it. If something about my algorithm is unclear, just ask. — Aziuth, Mar 04 '19 at 15:57
how is it possible to count the number of non zero pixels in a contour (not a bounding box) — clarkk, Mar 05 '19 at 09:59
@clarkk I don't understand your question in this context. What is a non zero pixel? A certain color? And why do you want to use a contour? Generally, what you would do in such a case, is to go from left to right, raise a counter whenever you cross a line of the contour and count each pixel as within the contour when your counter currently stores an odd number. — Aziuth, Mar 05 '19 at 10:21
non zero is a pixel that is not `0` (black).. The area that needs to be inverted doesn't have to be 100% rectangular.. Therefor I need all pixels inside a countour — clarkk, Mar 05 '19 at 10:23
@clarkk Or you simply follow my algorithm and invert the component that is found as a union of connected components. — Aziuth, Mar 05 '19 at 10:25
I still need to do some validation like the ratio between black and white pixels in the invert candidate areas.. — clarkk, Mar 05 '19 at 11:35
It also seems like I can't do any shape detection on connectedComponents.. It can only be done on contours — clarkk, Mar 05 '19 at 11:37
@clarkk Please read my algorithm again. We don't need any shapes. We create connected components. This is done by something like the BFS algorithm, and the result is not some sort of shape type but a list of pixel coordinates. And the ratio of black to white is irrelevant in my algorithm, all that matters is that the white surrounds the black, which can be checked as simple as comparing the minimal and maximal coordinates. Note that my algorithm does not rely on any library function within OpenCV, it just requires access to the pixels. — Aziuth, Mar 05 '19 at 12:07

detect areas with inverted background/foreground

1 Answers1