3

I have to understand if an image contains another similar image. Here 2 example:

Inside this image: enter image description here I need to find this image: enter image description here

or inside this enter image description here find this enter image description here.

The idea is: given an input image and a set of icons find which icon is present in the input image.

I've tried using MatchTemplate and feature matching with ORB and SIFT but I couldn't find any valid matches.

Here my try with MatchTemplate in Go:

package main

import (
    "fmt"
    "image/color"

    "gocv.io/x/gocv"
)

func main() {
    matImage := gocv.IMRead("/Users/pioz/Desktop/samplex.jpg", gocv.IMReadGrayScale)
    // gocv.Canny(matImage, &matImage, 200, 400)
    matTemplate := gocv.IMRead("/Users/pioz/Desktop/eld.jpg", gocv.IMReadGrayScale)
    // gocv.Canny(matTemplate, &matTemplate, 20, 40)
    matResult := gocv.NewMat()
    mask := gocv.NewMat()
    gocv.MatchTemplate(matImage, matTemplate, &matResult, gocv.TmCcoeffNormed, mask)
    mask.Close()
    minConfidence, maxConfidence, minLoc, maxLoc := gocv.MinMaxLoc(matResult)
    fmt.Println(minConfidence, maxConfidence, minLoc, maxLoc)

    gocv.Circle(&matImage, minLoc, 10, color.RGBA{0, 0, 255, 1}, 10)
    gocv.Circle(&matImage, maxLoc, 10, color.RGBA{0, 0, 255, 1}, 10)

    gocv.IMWrite("out/out.jpg", matImage)
}

Do you have any advice or snippet to solve this kind of problem?

Pioz
  • 6,051
  • 4
  • 48
  • 67
  • share some code. might be a bug in the writing. Or maybe the pipeline is missing some filters –  Oct 29 '19 at 09:25
  • @mh-cbon I've edited the question with my code. – Pioz Oct 29 '19 at 10:45
  • great. However none of it make use of sift or orb. and the template matching method as mentioned in another thread is not capable of handling scale issue. It is more like patch detection. as explained in https://stackoverflow.com/a/58160295/4466350 `Off the bat, template matching doesn't directly help you match things that are scaled, rotated, or warped. Template matching is strictly concerned with measuring the similarity of two images exactly as they appear.` –  Oct 29 '19 at 10:49
  • 1
    template matching is scale and rotation sensitive. It only looks for positional matches of the same size and orientation. So if the template and the region in the larger image are similar, but have different sizes, then template matching won't work unless you try all sizes of the template. There are special tools for doing scale and rotation invariant template matching. But I am not sure if they exist in OpenCV. – fmw42 Oct 29 '19 at 16:26
  • @fmw42 I've tried my example with the template of the same scale and orientation, but I couldn't find any valid match. – Pioz Oct 29 '19 at 16:55
  • `@Pioz` That is likely because of the color differences and contrast inversion (white letters on black vs black letters on blue). Perhaps doing the template matching on edge extracted images might work better. – fmw42 Oct 29 '19 at 17:43

2 Answers2

1

I believe this question has already been asked - here.

Templete matching is supposed to be one of the best techniques for this kind of image processing. So, if it's not working for you, try reviewing/sharing the code you have implemented.

Avinash
  • 133
  • 9
  • 2
    The answer to the question you have linked say: "This can be done with template matching. To summarize (my understanding), template matching looks for an _exact_ match of one image within another image." But in my case, it is not an exact match infact the icon I need to find is similar, not identical. – Pioz Oct 29 '19 at 09:09
  • note that you could try the template matching with various scales as described in https://www.pyimagesearch.com/2015/01/26/multi-scale-template-matching-using-python-opencv/ in your case that could do it. –  Oct 29 '19 at 09:28
  • @Pioz thing is, you have an advanced brain as a human and can identify things that are similar but not the same. But which parameters you use? If you use color, or if you use structure/edges, you will get different images. You need to *define* "similar" mathematically before you can find an algorithm that will work for you. – Ander Biguri Oct 29 '19 at 15:01
0

This link is related to OpenCV which explains feature matching and different techniques. There is also examples with code, which may come of use. Also if SIFT doesn't give staisfactory result, try affine SIFT or ASIFT.

Beeti Sushruth
  • 321
  • 2
  • 12