Project 5 Part A: IMAGE WARPING and MOSAICING

For this part of the project I learned how to recover homographies from pairs of images, and use that knowledge to rectify images and blend them into a mosaic.

Shooting Pictures

I and a couple of other students in the class went to take some photos, and while you will see more here are a couple of pairs of images that are connected by a projective transform along with their correspondence points:

Recover Homographies

Using the technique mentioned here, I was able to compute the homography matrix H that maps points p to points p’. For example, for the Sather Gate set above I was able to recover the following H matrix:

[[ 2.32919277e-03  8.71539605e-05  6.84444894e-01]

 [ 5.44256211e-04  8.47839003e-04 -7.29059278e-01]

 [ 2.46026308e-07  9.56601540e-09  1.14403012e-03]]

I compared the correctness of my H matrix with OpenCV’s findHomography function and found they were very comparable. However, when I tried to warp some points I encountered significant numerical issues when I normalized the matrix w.r.t. the bottom right most element, and had better results normalizing by the third coordinate of the point post-transform.

Image Warping

Once I was able to reliably compute H, I moved onto image warping, and used the H image I found above to inverse warp the entire left image onto the right image. Here are the results. The interpolation function I used is RectBivariateSpline:

Before

After

Image Rectification

Having built all the building blocks, I moved onto image rectification with other images in my set along with this one, by mapping a square region of the original image onto a square region on an empty canvas, and here are the images, their initial correspondence points, and the rectified images. I have also noted the canvas points I used, which I either estimated from a ground-truth image or if a square is known I just used that:

Image Stitching

Given a set of images, I used the techniques developed above to warp all into one image and then alpha-blended them at the intersections. Here are a couple of results:

(in-progress)