Is it possible to use vanishing points and gradient descent to align two images or conduct pose rectification?
The case is, I'm using SLAM and have a prior 3D model. I have two images A and B. Image A is obtained from SLAM and image B is obtained by inputting the SLAM pose in a prior model. But image B is not well aligned with image A. So my objective is to correct the SLAM pose in a prior model and get a better-aligned image in B.
I'm thinking, I detect three orthogonal vanishing points in two images. Define the horizontal and vertical vanishing lines. The intersected point for two lines is regarded as a finite vanishing point. So I have finite vanishing point for each of images A and B.
Now I define two loss functions: one is to measure the distance error between two images, and the other one is to measure the angle difference between two horizontal vanishing lines.
**Then conduct gradient descent to obtain a new pose to get a better image B. **
I'm quite not sure if this works and how the last step - gradient descent derives the new pose.
Hope my question makes sense and appreciate your help!!!
Houhao
I use pytorch sgd optimizer to conduct gradient descent.
three vanishing points in images A and B to have a horizontal and vertical line, finite vanishing point.
loss = distance_loss + angle_loss
update.
Then have three vanishing points. do previous before to get the finite vanishing point. estimate the rotation matrix and dot product to SLAM pose.
