Is there an algorithm that can take one image as an input and a list of images as second input and tell which one is the most similar ? In our problem we can have images which are the same buy look different due to watermarks. So we would need to identify matching images even when watermarks are different.
Are neural networks used for this? Is there a particular algorithm?
Keypoint extraction and matching is one of the solutions for this problem. Using some feature detector as SIFT, SURF, Fast-To-Track, ... to extract keypoints on original image and other images. On these days, SIFT detector was become popularly and be improved much because its accuracy and efficiency.
After that, features is matched by Ransac algorithm or others...
The matching images can be defined by a number of true matching points.
Can refer keypoint algorithm in OpenCV: API: http://docs.opencv.org/2.4/modules/refman.html Example:http://docs.opencv.org/2.4/doc/tutorials/features2d/table_of_content_features2d/table_of_content_features2d.html#table-of-content-feature2d
One possible approach is perceptual hashing, which is quite robust and can be tuned. Survey paper link
Of course this could also be done with Deep Learning, but this will need much more work. Slides
It will always be needed to define your problem-setting. Are there pixelwise-equal results (even classic hashing on pixel work); are there different compressions (robustness needed); are there color-transformations or even geometric-transformations (like rotations); size-changings... The classic perceptual-hashing algorithms are quite robust, even for moderate transformations.
Some time ago i implemented a simple example of a perceptual-frame hash (based on pixel-statistics, not features). I would try something like this first before "going deeper" (CNNs) :-)
There is also the question of the query-process. While i don't remember the hashing-properties of my approach (maybe a metric for bit-errors is enough), the Deep-Learning one in the Slides try to preserve similarities, so that you could get distances / a ranking while querying.
If you want to tackle with watermark attack, I will recommend you image hash algorithms, they are fast and robust to this kind of attack. Let me show you some examples
#include <opencv2/core.hpp>
#include <opencv2/core/ocl.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/img_hash.hpp>
#include <opencv2/imgproc.hpp>
#include <iostream>
void watermark_attack(cv::Ptr<cv::img_hash::ImgHashBase> algo)
{
std::vector<std::string> const origin_img
{
"origin_00.png", "origin_01.png",
"origin_02.png", "origin_03.png"
};
std::vector<std::string> const watermark_img
{
"watermark_00.png", "watermark_01.png",
"watermark_02.png", "watermark_03.png"
};
cv::Mat origin_hash, watermark_hash;
for(size_t i = 0; i != origin_img.size(); ++i){
cv::Mat const input = cv::imread(origin_img[i]);
cv::Mat const watermark_input = cv::imread(watermark_img[i]);
//compute the hash value of image without watermark
algo->compute(input, origin_hash);
//compute the hash value of image with watermark
algo->compute(watermark_input, watermark_hash);
//compare the different between the hash values
//of original image and watermark image
std::cout<<algo->compare(origin_hash, watermark_hash)
<<std::endl;
}
}
int main()
{
using namespace cv::img_hash;
//disable opencl acceleration may(or may not) boost up speed of img_hash
cv::ocl::setUseOpenCL(false);
watermark_attack(AverageHash::create());
}
The results are 1,2,1,2, all pass.
This small program compare the original image(left) with their watermark brother(right), the smaller the value of compute show, more similar the images are. For the case of AverageHash, recommend threshold is 5(that means if the compare result is bigger than 5, the images are consider very different).
Not only watermark, AverageHash provide another "side effects", this algorithm work under contrast , noise(gaussian, pepper and salt), resize and jpeg compression attack too.
The other benefit of using image hash is you can store the hash values of your images in a file, you do not need to compute the hash values again and again.
Simple and fast method to compare images for similarity will show you more details about the img_hash module of opencv.
Simplest approach would be registration (alignment) (ECC registration is present for both Matlab and OpenCV), you could devise your own means for how to create a scheme for this.
Better approach would be using MSER or FAST features present in Matlab. Use their documentation, they literally do what you ask.
PS: There's a cascade trainer inbuilt in Matlab 2015b which does just what you're asking. Takes a reference image, asks for background w/o reference image and WALLA you got yo own cascade classifier ready, use it at your leisure to classify images left and right, side and bottom, son.
