6

I am taking screenshots of the screen using robot and then searching for smaller images within those screenshots. This works on Windows but not OS X because of gamma correction. The best solution I can come up with is to simply match similar colours instead of exact color matches.

My fear is that matching similar colours will mean going beyond getRGB therefore will slow down my program (because it's taking multiple screenshots and comparing them to a smaller image to search for a match very quickly).

My question is, lets say I had BufferedImage Screenshot and BufferedImage smallImage, how would I go about determining if Screenshot.getRGB(1,1) and smallImage.getRGB(1,1) are similar colours?

Andrew Thompson
  • 168,117
  • 40
  • 217
  • 433
David Zorychta
  • 13,039
  • 6
  • 45
  • 81
  • you asked in a comment about a faster way to get pixel values than *getRGB*... *getRGB* can indeed be terribly slow: on some OS X versions it can be up to two orders magnitude (yup, a hundred time slower [!], measured on OS X 10.4) slower than directly getting the pixel values from the underlying *int[]*. See my (accepted) answer here about a faster way to extract an histogram from an image: http://stackoverflow.com/questions/8218072/faster-way-to-extract-histogram-from-an-image/8218570#8218570 A similar technique could be applied in your case to both be faster and dodge gamma correction. – TacticalCoder Dec 31 '11 at 15:53

3 Answers3

4

There's an interesting paper on exactly this problem:

A New Perceptually Uniform Color Space with Associated Color Similarity Measure for Content-Based Image and Video Retrieval by M. Sarifuddin and Rokia Missaoui

You can find this easily using Google or in particular Google Scholar.

To summarise, some color spaces (e.g. RGB, HSV, Lab) and distance measures (such as Geometric mean and Euclidean distance) are better representations of human perception of color similarity than others. The paper talks about a new color space, which is better than the rest, but it also provides a good comparison of the common existing color spaces and distance measures. Qualitatively*, it seems the best measure for perceptual distance using commonly available color spaces is : the HSV color space and a cylindrical distance measure.

*At least, according to Figure 15 in the referenced paper.

The cylindrical distance measure is (in Latex notation):

D_{cyl} = \sqrt{\Delta V^{2}+S_1^{2}+S_2^{2}-2S_1S_2cos(\Delta H)}

Also note there are some similar questions that address the same issue:

finding similar colors programatically

"Distance" between colours in PHP

Finally, in Java, there's any easy way to convert from RGB values into other color spaces:

ColorSpace.fromRGB

Community
  • 1
  • 1
Tim Gee
  • 1,062
  • 7
  • 9
1

For color similarity, let DR=R1-R2, DG=G1-G2, DB=B1-B2, then calculate DR×DR+DG×DG+DB×DB and take the square root of that, and compare it to some threshold value. Of course, this can be optimized by simply skipping the square root part and just squaring the threshold.

But you might be better off if you manage to somehow get OSX to render a PNG containing your image on-screen, then read that part of the screen, and then look for an exact match against that.

Mike Nakis
  • 56,297
  • 11
  • 110
  • 142
  • Yours is a geometric mean, I think. How is this better than using a Euclidean distance calculation? In pseudocode, `sqrt(sqr(r1-r2) + sqr(g1-g2) + sqr(b1-b2))` – Hovercraft Full Of Eels Dec 31 '11 at 15:00
  • Forgive me for asking, but is there a more efficient way of obtaining RGB values of a pixel coordinate other than breaking it up after calling getRGB()? I would love to know how to configure captureScreen in robot to render OS X screenshots correctly with a better gamma correction similar to Windows, but I do not know how to do this. – David Zorychta Dec 31 '11 at 15:00
  • @HovercraftFullOfEels damn Alzheimer's. I will correct my answer. – Mike Nakis Dec 31 '11 at 16:03
  • @MikeNakis: thanks. Just picture the results of your calc if one color is `[100, 200, 50]` and the other is `[100, 0, 255]`. It would calculate a "distance" of 0 since r1 - r2 is 0. – Hovercraft Full Of Eels Dec 31 '11 at 16:06
  • @user309641 once you have the pixel value in an int, splitting the int into R,G,B might look complicated because of all the shifts and the ands and what not, but computationally, it is one of those things that a CPU tends to do extremely fast and easy. I would, though, advise you to use the form of getRGB which returns an array of ints and not the one which returns just a single int. That's going to speed things up **a lot**. – Mike Nakis Dec 31 '11 at 16:11
  • It's the square-root which is computationally expensive, and quite possibly not necessary if all he wants is a quick and dirty calc. – Hovercraft Full Of Eels Dec 31 '11 at 16:47
0

are you using this getRGB Method? If yes I would compare the array elements 1 to 3 (RGB) with each other by subtracting them and checking if the absolute value of this is below a given threshold (e.g. 10). So it would be something like

if(Math.abs(firstPixel[1]-secondPixel[1])<10) 
   redIsSimilar = true; 
else 
   redIsSimilar=false;

without if statement:

redIsSimilar = Math.abs(firstPixel[1]-secondPixel[1]) < 10;

Of course you need to copy paste this for green and blue as well. Hope this helps, Christoph

dani herrera
  • 48,760
  • 8
  • 117
  • 177
ChristophMa
  • 129
  • 2
  • 2
  • 6
  • I am using getRGB of BufferedImage class here http://docs.oracle.com/javase/1.4.2/docs/api/java/awt/image/BufferedImage.html#getRGB(int, int) – David Zorychta Dec 31 '11 at 15:13