There a a few SO posts about image recognition floating around. This one is probably the closest match to what you want and Tom Gullen's answer is very comprehensive. You may also like to look at redmoskito's answer to this question.
One fairly basic method that I have seen that is not mentioned in either of these posts is the following:
(I can't take credit for this - if someone else can find the SO post this is from, please let me know!)
- Shrink the image you wish to compare to a small size (e.g. 4x4 px)
- Shrink the user's hand-drawn image to the same size
- Iterate over the pixels and compare their (for example, RGB) data to the shrunk reference image pixels.
- ((insert comparison threshold here)) if the individual pixels are 'similar enough' to the pixels of the original image - you have a match.
This can work fairly well if you have a closed set of comparison images and you know that one of those images will be the one drawn (the idea being that each image will have a unique 4x4px 'fingerprint').
Its overall effectiveness is based on the algorithm you use to determine what defines a "similar pixel" (e.g. similar RGB values, nearest-neighbour similarity, etc) and, naturally the larger your shrunk image is, the more exact the process will be. I have used this general procedure with reasonable success for recognising basic shapes and characters. You'll need a pretty good (and thoroughly tested) logic algorithm to make this production quality though.
As for your second question, I assume you mean how to remove fog that you trace your finger over (like what happens in real life). One way of achieving this would be to detect where the finger is and "draw an alpha channel" which then acts as a mask for your fog image. Or, you could draw directly to the image and set the relevant pixels' alpha values to 0.
These are just some ideas, the area of image comparison and manipulation is huge. But hopefully this will provide a starting point for further exploration.
EDIT:
Apple provide two nice (iOS-compatible) functions for extracting pixel data. If you wrap them in a function:
+ (NSMutableData *)pixelDataFromImage:(UIImage *)image {
NSMutableData *pixelData = (__bridge_transfer NSMutableData *)
CGDataProviderCopyData(CGImageGetDataProvider(image.CGImage));
return pixelData;
// Return data is of the form [RGBA RGBA RGBA ....]
// ^^^^ ^^^^ ^^^^
// Byte Index: 0123 4567 89..
// ^ ^ ^
// Pixel Number: px1 px2 px3
}
So to tie it together into a (minimal) algorithm based on the 4 steps above:
//...
NSMutableData *imagePixelData = [self pixelDataFromImage:image];
NSMutableData *referencePixelData = [self pixelDataFromImage:reference];
// Both image and reference are UIImages
if ([imagePixelData length] != [referencePixelData length]) {
return 0.0f; // Can't compare, different number of pixels
}
Byte *imagePixelBytes = [imagePixelData mutableBytes];
Byte *referencePixelBytes = [referencePixelData mutableBytes];
int totalDifference = 0;
float averageDifference = 0;
int bytesCompared = 0;
for (int i = 0; i < [imagePixelData length]; i++) {
if ((i+1) % 4 == 0) { // Compare only alpha values in this example
// (compares images ignoring colour)
int difference = (int)fabs(imagePixelBytes[i] - referencePixelBytes[i]];
totalDifference += difference;
bytesCompared += 1;
}
}
averageDifference = totalDifference/bytesCompared;
float similarity = 1.0f - (averageDifference/255);
return similarity;
// 1.0 => Exact match
// Now you need to determine a threshold for "how similar means 'the same'".
As I said this is only minimal, but it's one way of implementing the procedure outlined above. Certainly the two Core Graphics functions make life much easier, and once you have the data, you just end up comparing two byte-arrays. Note you'll still need to shrink the images first (with Core Graphics) - there are a few tutorials around for that (e.g. here).
