iOS UIImage Binarization for OCR - handling images with varying luminance

Question

I had a C++ binarization routine that I used for later OCR operation. However I found that it produced unnecessary slanting of text. Searching for alternatives I found GPUImage of great value and it solved the slanting issue.

I am using GPUImage code like this to binarize my input images before applying OCR.

However the threshold value does not cover the range of images I get. See two samples from my input images:

I can't handle both with same threshold value. Low value seems to be fine with later, and higher value is fine with first one.

The second image seems to be of special complexity because I never get all the chars to be binarized right, irrespective of what value I set for threshold. On the other hand, my C++ binarization routine seems to do it right, but I don't have much insights to experiment into it like simplistic threshold value in GPUImage.

How should I handle that?

UPDATE:

I tried with GPUImageAverageLuminanceThresholdFilter with default multiplier = 1. It works fine with first image but the second image continues to be problem.

Some more diverse inputs for binarization:

UPDATE II:

After going through this answer by Brad, tried GPUImageAdaptiveThresholdFilter (also incorporating GPUImagePicture because earlier I was only applying it on UIImage).

With this, I got second image binarized perfect. However first one seems to have lot of noise after binarization when I set blur size is 3.0. OCR results in extra characters added. With lower value of blur size, second image loses precision.

Here it is:

+(UIImage *)binarize : (UIImage *) sourceImage
{
    UIImage * grayScaledImg = [self toGrayscale:sourceImage];
    GPUImagePicture *imageSource = [[GPUImagePicture alloc] initWithImage:grayScaledImg];
    GPUImageAdaptiveThresholdFilter *stillImageFilter = [[GPUImageAdaptiveThresholdFilter alloc] init];
    stillImageFilter.blurSize = 3.0;    

    [imageSource addTarget:stillImageFilter];   
    [imageSource processImage];        

    UIImage *imageWithAppliedThreshold = [stillImageFilter imageFromCurrentlyProcessedOutput];
  //  UIImage *destImage = [thresholdFilter imageByFilteringImage:grayScaledImg];
    return imageWithAppliedThreshold;
}

Answer 1

For a pre processing step you need adaptive thresholding here.

I got these results using opencv grayscale and adaptive thresholding methods. Maybe with an addition of low pass noise filtering (gaussian or median) it should work like a charm.

I used provisia (its a ui to help you process images fast) to get the block size I need: 43 for the image you supplied here. The block size may change if you take photo from closer or further. If you want a generic algorithm, you need to develop one that should search for the best size (search until numbers are detected)

EDIT: I just saw the last image. It is untreatably small. Even if you apply the best pre-processing algorithm, you are not going to detect those numbers. Sampling up would not be solution since noises will come around.

Answer 2

I finally ended up exploring on my own, and here is my result with GPUImage filter:

+ (UIImage *) doBinarize:(UIImage *)sourceImage
{
    //first off, try to grayscale the image using iOS core Image routine
    UIImage * grayScaledImg = [self grayImage:sourceImage];
    GPUImagePicture *imageSource = [[GPUImagePicture alloc] initWithImage:grayScaledImg];
    GPUImageAdaptiveThresholdFilter *stillImageFilter = [[GPUImageAdaptiveThresholdFilter alloc] init];
    stillImageFilter.blurSize = 8.0;

    [imageSource addTarget:stillImageFilter];
    [imageSource processImage];

    UIImage *retImage = [stillImageFilter imageFromCurrentlyProcessedOutput];
    return retImage;
}

+ (UIImage *) grayImage :(UIImage *)inputImage
{    
    // Create a graphic context.
    UIGraphicsBeginImageContextWithOptions(inputImage.size, NO, 1.0);
    CGRect imageRect = CGRectMake(0, 0, inputImage.size.width, inputImage.size.height);

    // Draw the image with the luminosity blend mode.
    // On top of a white background, this will give a black and white image.
    [inputImage drawInRect:imageRect blendMode:kCGBlendModeLuminosity alpha:1.0];

    // Get the resulting image.
    UIImage *outputImage = UIGraphicsGetImageFromCurrentImageContext();
    UIGraphicsEndImageContext();

    return outputImage;
} 

I achieve almost 90% using this - I am sure there must be better options but I tried with blurSize as far as I could and 8.0 is the value that works with most of my input images.

For anyone else, good luck with your trying!

Answer 3

SWIFT3

SOLUTION 1

extension UIImage {

func doBinarize() -> UIImage? {

    let grayScaledImg = self.grayImage()
    let imageSource = GPUImagePicture(image: grayScaledImg)
    let stillImageFilter = GPUImageAdaptiveThresholdFilter()
    stillImageFilter.blurRadiusInPixels = 8.0 

    imageSource!.addTarget(stillImageFilter)
    stillImageFilter.useNextFrameForImageCapture()
    imageSource!.processImage()


    guard let retImage: UIImage = stillImageFilter.imageFromCurrentFramebuffer(with: UIImageOrientation.up) else {
        print("unable to obtain UIImage from filter")
        return nil
    }

    return retImage
}

func grayImage() -> UIImage? {
    UIGraphicsBeginImageContextWithOptions(self.size, false, 1.0)
    let imageRect = CGRect(x: 0, y: 0, width: self.size.width, height: self.size.height)

    self.draw(in: imageRect, blendMode: .luminosity, alpha:  1.0)

    let outputImage = UIGraphicsGetImageFromCurrentImageContext()
    UIGraphicsEndImageContext()

    return outputImage
}


}

The result would be

SOLUTION 2

use GPUImageLuminanceThresholdFilter to achieve 100% black and white effect whithout grey color

   let stillImageFilter = GPUImageLuminanceThresholdFilter() 
   stillImageFilter.threshold = 0.9 

For example I need to detect flash light and this works for me