0

In my previous question I was trying to match multispectral images from satellite images to create a 4 channel image (R,G,B,IR). It seemed that ECC was not the right way to go. A comment suggested using Feature-based matching, something I was in the process of trying while waiting for some replies on the previous question.

The results are... absolutely stunning to me. I hope you guys can help me find my mistake since this, IMHO, has to be a configuration problem on my side. SIFT can't be performing that badly.

Setup

I used the python code from this guide. Only lines that changed are line 15 and 20, they are:

 orb = cv2.SIFT_create(MAX_FEATURES) #line 15
 matcher = cv2.BFMatcher(cv2.NORM_L2,crossCheck=False) #line 20

I intentionally did not change the name orb in order to keep the rest of the code as close to the original as possible. So we are ready to overlay the images. Let's go! All images I used are Satellite Images from SentinelHub. It is expected that Images are mostly rotated and translated towards each other, no change in perspective shall be considered here (thus a Euclidian Motion is expected)

Usage and Results

Initially, I used these images (all images in this post will be rather small because I will use quite a few): The IR Image is in fact slightly rotated so the result looks good!

IR Image 1 TC Image 1 Aligned Imate 1

So far so good right? Well... no. Look at the matches. It really seems like the luck of a million years aligns these two images. These are the 15% best matches!!!!

Matches of Image 1

What do you do at this point? Rerun with different images.

So I did with the following set of images and I saw what I expected. (In short just the matches and the alignment)

Matches Image 2 Alignment Image 2

I expected this to happen after the first set of images. Keep in mind these are again the 15% best matches (I tried different numbers, didn't change anything)

To check my sanity I then also tried to match the red channel with the rest of the RGB image. That worked and looked as expected. I even spun the Image slightly to throw a little challenge at it and it worked.

Matching R Channel to RGB

Aftermath Sooo... I did this with more images than you can see here. I always used Images of lakes and lakes shores because I think, for a features-based matching algorithm, a shoreline should provide a great baseline. Somehow that does not seem to help at all. I really don't know what to do or say at this point because the images are full of features that, for a human, are easy to correlate. However, it seems that SIFT has an insanely hard time matching RGB to IR images and I don't really understand why.

Questions

  1. Any ideas about what's wrong here?
  2. Is SIFT not suited for this use case?
  3. Is something configured the wrong way?

Edit: I have been asked to add on some images that weren't matchable, so here they come :) The previous ones were not uploaded due to them breaching the 2MB limit. You can obtain your own here. Short tutorial:

  1. Take TrueColor Screenshot (yes screenshot for now)
  2. Click on "Custom"
  3. Choose all Channels as "8" (its drag and drop)
  4. Take another screenshot of roughly the same area

enter image description here enter image description here

Clex
  • 43
  • 8
  • I dont think SIFT is suited for that task unless you can show that both images' textures have similar gradients in all the regions. You might need some kind of edge/shape-matching instead. – Micka Jul 05 '21 at 12:52
  • For SIFT and other texture methods you need the images to be in the same modality, so that the textures look the same. If you want to match coastlines, extract coastlines and match them separately from the land textures. – Cris Luengo Jul 05 '21 at 14:04
  • Just in case someone tries to post an answer... I suggest you to post few clean images without the matching lines. The lines degrade the images and make them inadequate for matching. – Rotem Jul 05 '21 at 14:44
  • SIFT is fast enough and generally provides good results. In your case, you need a better feature description algorithm. See [SURF results in the previous question](https://stackoverflow.com/a/68255635/12447766). – Burak Jul 06 '21 at 12:52
  • 1. You could try to load your RGB image as grayscale image. 2. I would recommend a different feature matching method e.g. SURF, RIFT, D2-net ... – Grillteller Jul 06 '21 at 13:00
  • 1
    @Grillteller they are loaded in the code in grayscale (just look at the example code) 2. thanks I am in the process of getting there, I found some promising results for now I will probably be able to answer my own question in a few days. – Clex Jul 08 '21 at 07:35

0 Answers0