OpenCV and Python -- How to superimpose images by specifying coordinates?

Question

In short, my question is how do I put an image on top of another by specifying specific coordinates for the added image? I would need to extend the "canvas" of the base image as needed so that the added image doesn't get cropped.

Here's the extended version:

My project is to take pictures extracted from a drone video and make a rough map with them, by aligning one photo with the last. I know there is software I can use to do this, like Agisoft Photoscan, but my goal is to create a more lightweight, rough solution.

So here's my plan, which I intend to do with each frame:

1. Use estimateRigidTransform, to generate the transformation matrix to align curr_photo with the last photo, base
2. Calculate the bounding rectangle needed to enclose the resulting image (using transformations of the four corners)
3. Modify the transformation matrix so that the top left of the bounding box is at the origin
4. Apply the transformation to the current photo, using the bounding rectangle's width and height to ensure none of the resulting image gets cropped
5. Super-impose the current image with the last image (making sure no cropping of either image occurs), by adding curr_image to base at the proper coordinates. This step is what I am asking about.

Here is the code that does steps one to four.

import numpy as np
import cv2


base = cv2.imread("images/frame_03563.jpg")
curr_photo = cv2.imread("images/frame_03564.jpg")

height, width = curr_photo.shape[:2]

# Step 1
# which transformation is required to go from curr_photo to base?
transformation = cv2.estimateRigidTransform(curr_photo, base, True)

# Step 2
# add a line to the affine transformation matrix so it can be used by
# perspectiveTransform
three_by_three = np.array([
    transformation[0],
    transformation[1],
    [0, 0, 1]], dtype="float32")

# get corners of curr_photo (to be transformed)
corners = np.array([
    [0, 0],
    [width - 1, 0],
    [width - 1, height - 1],
    [0, height - 1]
])

# where do the corners of the image go
trans_corners = cv2.perspectiveTransform(np.float32([corners]), three_by_three)

# get the bounding rectangle for the four corner points (and thus, the transformed image)
bx, by, bwidth, bheight = cv2.boundingRect(trans_corners)

# Step 3
# modify transformation matrix so that the top left of the bounding box is at the origin
transformation[0][2] = transformation[0][2] - bx
transformation[1][2] = transformation[1][2] - by

# Step 4
# transform the image in a window the size of its bounding rectangle (so no cropping)
mod_curr_photo = cv2.warpAffine(curr_photo, transformation, (bwidth, bheight))

# for viewing
cv2.imshow("base", base)
cv2.imshow("current photo", curr_photo)
cv2.imshow("image2 transformed to image 1", mod_curr_photo)

cv2.waitKey()

I've also attached two sample images. I used the first one as the base, but it works either way.

Answer 1

Edit: I have now turned the answer linked below into a Python module, which you can now grab from GitHub here.

---

I answered this question a few weeks ago. The answer should contain everything needed to accomplish what you're after; the only thing I don't discuss there is alpha blending or other techniques to blend the borders of the images together as you would with a panorama or similar.

In order to not crop the warped photo you need to calculate the needed padding beforehand because the image warp itself could reference negative indices, in which case it won't draw them...so you need to calculate the warp locations first, pad your image enough to account for those indices outside your image bounds, and then modify your warp matrix to add those translations in so they get warped to positive values.

This allows you to create an image like this:

Image from Oxford's VGG.