How to use Opencv to un-fade faded characters

Question

Hello Everyone I'm trying to pre-process some license plate images for an OCR task, While I have some good results in some plates, others end up getting faded characters, I believe that with opencv I can unfade these characters and tried things like dilate, but on the characters that were not faded, this operation ended up hurting the end-result of the good ones. Does anyone knows what is the best strategy here? My goal is to use a generic preprocess function for all cases.

Below a snippet of the code that I've been using to preprocess the images and good and bad I/O results.

def image_preprocessing(image_path):
    img = cv2.imread(image_path)
    height, width, _ = img.shape
    img = cv2.resize(img, (width*6, height*6), interpolation = cv2.INTER_LINEAR)
    img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    img = cv2.GaussianBlur(img, (1, 1), 0)
    img = cv2.adaptiveThreshold(img, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY_INV, 21, 3)
    img = cv2.bitwise_not(img)
    img = cv2.resize(img, (width, height), interpolation = cv2.INTER_AREA)
    img = cv2.GaussianBlur(img, (3, 3), 0)
    return img

Good Result Input:

Good Result Output:

Bad Result Input:

Bad Result Output:

I've tried to use dilate and erode with inverse and original threshold images, but could not get a good generic result.

@CrisLuengo I've applied now it if the help of this (https://stackoverflow.com/questions/42257173/contrast-stretching-in-python-opencv), it actually helped a lot with all the observed cases, it is not full black but helped — pydoni, Jul 28 '23 at 17:43

Ori Yarden PhD · Accepted Answer · 2023-07-28T21:58:23.557

The resolution of the second example image is incredibly low in terms of the number of pixels and in terms of its contrast:

And it's possible that images can be below the threshold resolution for any sort of computer vision to work.

Another way would be to process the image through a kernel filter and then set a threshold for the pixel values in another numpy array of zeros and populate it with ones where it crosses the threshold:

from skimage import io as io
image1 = image_path=io.imread('https://i.stack.imgur.com/FDN4R.jpg')

import numpy as np
def normalize_rgb_values(rgb_values, max_value=1.0):
    norm_rgb_values = (rgb_values - np.mean(rgb_values)) / np.var(rgb_values)**0.5
    norm_rgb_values += abs(np.min(norm_rgb_values))
    norm_rgb_values *= (max_value / np.max(norm_rgb_values))
    return np.round(norm_rgb_values, decimals=0).astype(int) if max_value == 255 else np.round(norm_rgb_values, decimals=9).astype(float)

_kernel = np.diag([-5.0, 1.0, 1.0])
from matplotlib import pyplot as plt
plt.imshow(_kernel)
plt.show()

_image1 = np.zeros((image1.shape[0] + _kernel.shape[0]*2, image1.shape[1] + _kernel.shape[1]*2, 3)).astype(float)
_image1[_kernel.shape[0]:-_kernel.shape[0], _kernel.shape[1]:-_kernel.shape[1], :] = image1
image1 = normalize_rgb_values(_image1, max_value=255)
plt.imshow(image1)
plt.show()

def filter_image(image1, _kernel):

    def yield_row_col_rgb_sums(image1, _kernel):
        for row in range(image1.shape[0] - _kernel.shape[0]):
            for col in range(image1.shape[1] - _kernel.shape[1]):
                for rgb in range(3):
                    yield np.sum(image1[row:row + _kernel.shape[0], col:col + _kernel.shape[1], rgb]*_kernel)

    return np.reshape(list(yield_row_col_rgb_sums(image1, _kernel)), [image1.shape[0] - _kernel.shape[0], image1.shape[1] - _kernel.shape[1], image1.shape[2]])

processed_image = normalize_rgb_values(filter_image(image1, _kernel))
plt.imshow(processed_image)
plt.show()

final_image = np.zeros((processed_image.shape[0], processed_image.shape[1])).astype(float)
final_image[np.where(np.sum(processed_image, axis=2) >= 1.385)] = 1.0
plt.imshow(final_image)
plt.show()

black_and_white_final_image = final_image
black_and_white_final_image.shape += (1,)
black_and_white_final_image = np.concatenate([black_and_white_final_image for _ in range(3)], axis=2)
plt.imshow(black_and_white_final_image)
plt.show()

Outputs:

EDIT for removing background

If we have additional parameters and fine tune the summed values threshold from the processed image a bit then we can get rid of (most of) the background:

final_image = np.zeros((processed_image.shape[0], processed_image.shape[1])).astype(float)
final_image[np.where(np.sum(processed_image, axis=2) >= 1.385)] += 1.0
final_image[np.where(np.sum(processed_image, axis=2) >= 1.75)] += 1.0
final_image[np.where(final_image == 2.0)] = 0.0
plt.imshow(final_image)
plt.show()

black_and_white_final_image = final_image
black_and_white_final_image.shape += (1,)
black_and_white_final_image = np.concatenate([black_and_white_final_image for _ in range(3)], axis=2)
plt.imshow(black_and_white_final_image)
plt.show()

Outputs:

Thank you for the help! surely it is an improvement, by any chance it is possible to remove these 2 yellow borders? I've tried using sobel but it ends up affecting the letters, is there a way to see these borders, like getting the area of connected same color pixels and switching the color of the ones that has a minimum area? — pydoni, Jul 28 '23 at 17:27
I updated the answer to include a black-and-white version of the output image. Yes you could use another processing method or two to try to remove the borders based on whether there are connecting/neighboring pixels (e.g. https://stackoverflow.com/questions/75844086/how-to-group-drivable-areas-in-yolop/75844595#75844595) or based on color. If you KNOW the border will be on the edges of the image you could use like `outline_color = np.array([1.0, 1.0, 1.0]) - image[0, 0 :]` then subtract `outline_color` from image or something. — Ori Yarden PhD, Jul 28 '23 at 17:34
This might help (https://github.com/OriYarden/Object-Detection-Image-Processing-Machine-Learning-from-Scratch-in-Python-using-Numpy-Arrays) since I kinda cover some of those topics you're asking. You'd most likely have to use a combination of your method, my method(s), etc. — Ori Yarden PhD, Jul 28 '23 at 17:35
@pydoni I got rid of the background in the output image in my edit. — Ori Yarden PhD, Jul 28 '23 at 21:59

How to use Opencv to un-fade faded characters

1 Answers1