8

I have researched all over in order to find a kernel that performs adaptive thresholding on iOS. Unfortunately I do not understand the kernel language or the logic behind it. Below, I have found a routine that performs thresholding (https://gist.github.com/xhruso00/a3f8a9c8ae7e33b8b23d)

static NSString * const kKernelSource = @"kernel vec4 thresholdKernel(sampler image)\n"
"{\n"
"  float inputThreshold = 0.05;\n"
"  float pass = 1.0;\n"
"  float fail = 0.0;\n"
"  const vec4   vec_Y = vec4( 0.299, 0.587, 0.114, 0.0 );\n"
"  vec4  src = unpremultiply( sample(image, samplerCoord(image)) );\n"
"  float Y = dot( src, vec_Y );\n"
"  src.rgb = vec3( compare( Y - inputThreshold, fail, pass));\n"
"  return premultiply(src);\n"
"}";

Is it possible to rewrite this into an adaptive thresholding kernel? The image I am supplying to it has been turned into B&W and has already been blurred. Are there any resources you could point me to? I would like to stick with CoreImage as my whole stack is built around it.

Edit: The best example / reference from what I am trying to achieve has been implemented in GPUImage's GPUImageAdaptiveThresholdFilter - https://github.com/BradLarson/GPUImage/blob/c5f0914152419437869c35e29858773b1a06083c/framework/Source/GPUImageAdaptiveThresholdFilter.m

mmackh
  • 3,550
  • 3
  • 35
  • 51
  • I updated my code with description, metal and modern approach https://gist.github.com/xhruso00/a3f8a9c8ae7e33b8b23d I inspired myself with below code but the code below doesn't respect alpha premultiplication (transparent areas will become black or white). – Marek H Oct 17 '19 at 21:18

3 Answers3

6

How does this look: I've used the CoreImage CIBoxBlur (although the dedicated convolution filters may be faster) and passed the output of that into my existing threshold filter.

class AdaptiveThresholdFilter: CIFilter
{
    var inputImage : CIImage?


    var thresholdKernel =  CIColorKernel(string:
    "kernel vec4 thresholdFilter(__sample image, __sample threshold)" +
    "{" +
    "   float imageLuma = dot(image.rgb, vec3(0.2126, 0.7152, 0.0722));" +
    "   float thresholdLuma = dot(threshold.rgb, vec3(0.2126, 0.7152, 0.0722));" +

    "   return vec4(vec3(step(imageLuma, thresholdLuma)), 1.0);" +
    "}"
    )


    override var outputImage: CIImage!
    {
        guard let inputImage = inputImage,
            thresholdKernel = thresholdKernel else
        {
            return nil
        }

        let blurred = inputImage.imageByApplyingFilter("CIBoxBlur",
               withInputParameters: [kCIInputRadiusKey: 9])

        let extent = inputImage.extent
        let arguments = [inputImage, blurred]

        return thresholdKernel.applyWithExtent(extent, arguments: arguments)
    }
}

I found this image of a shaded page and with this code:

let page = CIImage(image: UIImage(named: "son1.gif")!)

let filter = AdaptiveThresholdFilter()

filter.inputImage = page

let final = filter.outputImage

I got this result:

enter image description here

Cheers!

Simon

Flex Monkey
  • 3,583
  • 17
  • 19
  • Thank you for getting me started. I was looking for something more along the lines of http://homepages.inf.ed.ac.uk/rbf/HIPR2/images/son1adp2.gif however – mmackh Mar 24 '16 at 11:45
5

Simon's Filter is the right approach to achieve the desired effect, however, you have to modify a couple of things.

First of all, switch the order of imageLuma and thresholdLuma, since we want black letters to remain black and not the other way around. Also, you should add a constant (I chose 0.01) to remove noise.

    var thresholdKernel =  CIColorKernel(string:
    "kernel vec4 thresholdFilter(__sample image, __sample threshold)" +
        "{" +
        "   float imageLuma = dot(image.rgb, vec3(0.2126, 0.7152, 0.0722));" +
        "   float thresholdLuma = dot(threshold.rgb, vec3(0.2126, 0.7152, 0.0722));" +
        "   return vec4(vec3(step(thresholdLuma, imageLuma+0.001)), 1);"     
    "}"

override var outputImage: CIImage! {
    guard let inputImage = inputImage,
        let thresholdKernel = thresholdKernel else {
        return nil
    }
    let blurred = inputImage.applyingFilter("CIBoxBlur", withInputParameters: [kCIInputRadiusKey: 5]) // block size
    let extent = inputImage.extent
    let arguments = [inputImage, blurred]
    return thresholdKernel.apply(withExtent: extent, arguments: arguments)
}

And this is, what you get Only using Apple's Core Image, without having to install any external libraries :)

enter image description here

Of course, you can play around a little with the values of constant and block size.

Pochi
  • 13,391
  • 3
  • 64
  • 104
Pascal
  • 2,590
  • 3
  • 21
  • 46
  • Pascal, thank you for your answer. Any idea on how to implement this in CoreImage too? https://github.com/ctodobom/OpenNoteScanner/blob/master/app/src/main/java/com/todobom/opennotescanner/ImageProcessor.java#L409 – mmackh Jun 02 '17 at 18:23
  • You mentioned a constant of 0.01 but you have 0.001, is this a mistake? Also, do you apply this constant to the imageLuma or the threshold luma (assuming i did want white letters with black background. – Pochi Jul 28 '17 at 03:29
  • 0.01 and 0.001 doesn't seem to make too much of a difference. You may try different values to fit your need. – triiiiista Dec 11 '17 at 05:12
4

You can use CIColorThresholdOtsu core image filter

  • 1
    This more recent answer is quite useful, too! If a developer writes a thresholding algorithm and just wants to apply the threshold, then CIColorThreshold is also worth a look: https://cifilter.io/CIColorThreshold/ – Rethunk Mar 29 '21 at 18:27