Image Warping and Mosaicing

By Jay Shenoy

Part 1: Shooting the Photographs

I shot three photos of the Berkeley skyline:

Image 1
Image 2
Image 3

Part 2: Recover Homographies

For the time being, I tried aligning only image 1 and image 2 above for simplicity. Here are the keypoints I selected for the images:

Image 1 Keypoints
Image 2 Keypoints

To recover the homography matrix between these two sets of keypoints, I solved the following system of linear equations using least-squares:

\( \begin{bmatrix} x_1 & y_1 & 1 & 0 & 0 & 0 & -x_1 x_2 & -y_1 x_2 \\ 0 & 0 & 0 & x_1 & y_1 & 1 & -x_1 y_2 & -y_1 y_2 \\ & & & & \vdots \\ \end{bmatrix} h = \begin{bmatrix} x_2 \\ y_2 \\ \vdots \end{bmatrix} \)

Here, \( (x_1, y_1) \) and \( (x_2, y_2) \) are corresponding keypoints from images 1 and 2, respectively, and there are 6 such correspondences. Least-squares lets you solve for the vector \( h \) that minimizes the squared error. \( h \) contains 8 entries that correspond to the first 8 entries of the homography matrix (the last entry is set to one).

Part 3: Warp the Images

Homography matrix in hand, I now warped image 1 to align properly with image 2. I did this using inverse warping: for each pixel in the warped image, I determined the source pixel in the original image and used interpolation to retrieve the proper pixel intensity. Image 1 was warped to align with image 2, and image 2 was translated to fit within a larger canvas (so that both images could be overlayed). The results of the individual warps are shown below:

Image 1 Warped
Image 2 Warped

Part 5: Blending

To properly mosaic the two warped images, I used a one-shot procedure that computed alpha weights for each pixel in each warped image. These alpha weights range from zero to one, indicating how much to feather each pixel value. The specific weights were computed linearly by setting the alphas to one at the centers of the unwarped images and having them decline to zero at the left and right edges. Below, you can see the feathered images as well as the final mosaic.

Image 1 Warped
Image 2 Warped
Mosaic