3

I'm working on ways of collecting farm aerial images (images collected from a helicopter in a perpendicular fashion) that I'd want to stitch them together to build the whole photo of the area that's being covered and then I wanted to run analytics. I'm assuming the images will come with [latitudue, longitude] coordinates, to help me determine the spots to place the images.

To understand the issues with this technology, I tried manually stitching pictures taken from my phone of some sample area in my back yard. I experienced that the edges don't usually look the same because they are being seen by the camera from different sides or angles. I guess this is a distortion in image that could potentially be fixed by ortho-rectification (not completely sure). I quickly created the following picture to help explain my problem.

enter image description here

My question to you:

What are the algorithms/techniques used to do ortho-rectification?

What tools would best suit my needs: opencv, or processing or matlab or any other tool that could easily help in rectification of images and creating a mosaic photo?

What other issues should be considered in doing aerial imagery mosaic-ing and analytics?

Thank you!

Rose
  • 2,792
  • 4
  • 28
  • 42
  • This question may be asked in [Geographic Information Systems](http://gis.stackexchange.com). An example: http://gis.stackexchange.com/questions/72062/how-to-merge-several-google-map-images-into-one – marsei Mar 04 '14 at 22:46
  • 3
    This question appears to be off-topic because it is about technical knowledge that isn't directly related to programming. – Andrey Rubshtein Mar 05 '14 at 09:13

3 Answers3

5

Image stitching usually assumes that the camera center is fixed across all photos, and uses homographies to transform the images so that they seem continuous. When the fixed camera center assumption is not strictly valid, artifacts/distortions may appear due to the 3D of the scene. If the camera center moved by a small distance compared to the relief of the scene, "seamless image blending" techniques may be sufficient to blur out the distortions.

In more extreme cases, ortho-rectification is required. Ortho-rectification (Wikipedia entry) is the task of transforming an image observed from a given perspective camera into an orthographic (Wikipedia entry) and usually vertical point of view. The orthographic property is interesting because it makes the stitching of several images much easier. The following picture from Wikipedia is particularly clear (left is an orthographic or directional projection, right is a perspective or central projection):

enter image description here

The task of ortho-rectification usually requires having a 3D model of the scene, in order to map appropriately intensities observed by the perspective camera to their location with respect to the orthographic camera. In the context of aerial/satellite images, Digital Elevation Models (DEM) are often used for that purpose, but generally have the serious drawback of not including man-made structures (only Earth relief). The NASA provides freely the DEM acquired by the SRTM missions (DEM link).

Another approach, if you have two images acquired at different positions, you could try to do a 3D reconstruction using one of the stereo matching technique, and then to generate the ortho-rectified image by mapping the two images as seen by a third orthographic and vertical camera.

OpenCV has several interesting function for that purpose (e.g. stereo reconstruction, image mapping functions, etc) and might be more appropriate for intensive usage. Matlab probably has interesting functions as well, and might be more appropriate for quick tests.

BConic
  • 8,750
  • 2
  • 29
  • 55
  • 1
    OpenCV's stitcher assumes true rotation (camera doesn't translate and only rotates around its axis), which automatically makes it unusable if translation occurs between images. I know that not only from personal experience but also from examining the source code of the stitcher class in this library. A custom routine has to be developed for such cases. – rbaleksandar Apr 28 '14 at 17:12
4

First, rectification is some kind of warping, but not the one that you need. Regular rectification is used in stereo to ensure that matching points lie on the same row - not your case. Ortho-rectification warps perspective projection into orthographic - again not your case. Not only you lack a 3D model for this warping to calculate but also you dont need it since your perspective distortions are negligible and you already have image pretty close to ortho ( that is when the size of the objects is small compared to the viewing distance perspective effects are small).

You problems in aligning two images stem from small camera rotations between shots. To start fixing the problem you need to ensure that your images actually overlap by say 30%. To read about this see chapter 9 of this book.

What you need is to review a regular image stitching techniques that use homography to map two images. Note that doing so assumes that images are essentially flat. To find homography you can first manually select 4 points in one image and 4 matching points in another image and run openCV function findHomography(). Note that overlap is required to find the matches (in your picture there is no overlap). warpPerspective() can warp images for you after homogrphy is found.

Vlad
  • 4,425
  • 1
  • 30
  • 39
  • DEMs (digital elevation models) can be acquired nowadays using techniques such as structure from motion (SfM) if enough image data is provided. Other than that some can be found online. – rbaleksandar Apr 28 '14 at 17:10
  • SFM is a super expensive algorithm with no solid guarantee on accuracy and seems to be an overkill for this case. It also addresses a general case of 3D structure rather than flat objects. Moreover, it simply won’t work with flat images since a fundamental matrix (a first step in SFM) won’t be sampled with representative points. – Vlad May 19 '15 at 20:16
1

"What are the algorithms/techniques used to do ortho-rectification?

If you want a good overview of the techniques, then the book "Multiple View Geometry in Computer Vision" by Hartley & Zisserman might be a good place to start: http://www.robots.ox.ac.uk:5000/~vgg/hzbook/

Andrew Zisserman also has some tutorials available here at www.robots.ox.ac.uk/~az/tutorials/ which might be more accessible/make it easier for you to find the particular technique you want to use.

"What tools would best suit my needs: opencv, or processing or matlab or any other tool that could easily help in rectification of images and creating a mosaic photo?"

OpenCV has a fair few number of tools available - take a look at Images stitching for starters. There's also a lot available for correcting distortion. However, it doesn't have to be the tool you use, there are others!

dr.blochwave
  • 161
  • 2
  • 7
  • For example there is this question with some good links on image rectification in OpenCV: http://stackoverflow.com/questions/4260594/image-rectification – dr.blochwave Mar 05 '14 at 08:38
  • Image rectification and ortho-rectification are two different things. – BConic Mar 05 '14 at 09:03