CS194-26 | Amy Huang
[Auto]Stitching Photo Mosaics

Part 1: Image Warping & Mosaicing

Part 2: Feature Matching for Autostitching

Part 1: Overview

In this project I compute homographies to warp images into image mosaics. Specifically I use registering, projective warping, resampling, and compositing these images in order to create a final 'panorama'-like piece. The first part of this is rectifying prospective images (like my book and laptop) into a front-view.

Recover Homographies

H = computeH(im1_pts, im2_pts)

I first needed to recover the parameters of the transformation between a pair of images. I used the equation: p' = Hp

The matrix H is the hommography while the p is the original image point. We use 8 degrees of freedom and we have at least 4 corresponodences. I use least squares to solve for this. To pick the corresponding values for the rectification, I chose 4 corners of a rectangle that is in a different perspective in my photo, and manually created a square shape for the im_2pts. For the mosaic, I chose points in the 2 (or more) photos that have overlapping features.

Warp the Images

warpedIm = warpImage(im, H)

For warping the images using the homography matrix, I first multiplied the original image corners with the homography to find the new shape of the warped image. That became my bounding box, and I was able to use forward warping and interpolated the value of P prime (the warped image). I did this on the separate color channels to produce a color image.

Image Rectification

With image rectification, the goal was to use images with planar surfaces, and warping them so that the plane is frontal-panal. For the first photo, I picked the four corners of my laptop's trackpad, and warped the image so that this trackpad would be perfectly rectangular, and frontal-panal perspective.

For my second photo, I repeated the same process except I used a book cover. As you can see, the cover of the book almost looks like it is viewed from the from view. A slight problem with using this is that the sides of the book shouldn't be still visible from the front, but there is not 3 dimensional information about the picture. The blue is the rectangle I warped to, and the red points are the points I selected for warping.

Blend the Images into a Mosaic

The next step was selecting corresponding points in separate images using cpselect, and warping one image to fit the points on the other. To do this, I first took the four corner points on the to-be-warped image, and used the homography matrix on that to find the bounding box dimensions. Then, I use forward warping on that image to fit into the polygon created using the bounding box.

warpImageMosaic(im, H)

Here are the images I took for the mosaic:

After warping the 2nd image to the first, I calculated the padding and applied that to the 1st image to have a good alignment. Here is what it looks like with them padded, and also with warped image placed on top.

With the new warping and alignment, I was able to blend the images together.

I used this same technique for another 2 images as well. Here is the before and after:

The most coolest thing I learned is probably using linear algebra concepts to stitch together images like panorama! I always loved using panorama and it's really cool how warping images can change perspectives so easily!

Part 2: Feature Matching for Autostitching