Project 5 Part A: IMAGE WARPING and MOSAICING
For this part of the project I learned how to recover homographies from pairs of images, and use that knowledge to rectify images and blend them into a mosaic.
I and a couple of other students in the class went to take some photos, and while you will see more here are a couple of pairs of images that are connected by a projective transform along with their correspondence points:
Using the technique mentioned here, I was able to compute the homography matrix H that maps points p to points p’. For example, for the Sather Gate set above I was able to recover the following H matrix:
[[ 2.32919277e-03 8.71539605e-05 6.84444894e-01]
[ 5.44256211e-04 8.47839003e-04 -7.29059278e-01]
[ 2.46026308e-07 9.56601540e-09 1.14403012e-03]]
I compared the correctness of my H matrix with OpenCV’s findHomography function and found they were very comparable. However, when I tried to warp some points I encountered significant numerical issues when I normalized the matrix w.r.t. the bottom right most element, and had better results normalizing by the third coordinate of the point post-transform.
Once I was able to reliably compute H, I moved onto image warping, and used the H image I found above to inverse warp the entire left image onto the right image. Here are the results. The interpolation function I used is RectBivariateSpline:
Before | After |
Having built all the building blocks, I moved onto image rectification with other images in my set along with this one, by mapping a square region of the original image onto a square region on an empty canvas, and here are the images, their initial correspondence points, and the rectified images. I have also noted the canvas points I used, which I either estimated from a ground-truth image or if a square is known I just used that:
Given a set of images, I used the techniques developed above to warp all into one image and then alpha-blended them at the intersections. Here are a couple of results:
(in-progress)