CS194-26 Project 4A: Image Warping and Mosaicing

Matthew Tang

Part 1: Shoot and digitize pictures

The first step was to take pictures. I used my iPhone7 to take the photos, trying my best to only rotate the angle and not translate the reference point (to be able to accurately reconstruct the homographies later). I took photos for 3 sets of panoramas.

Campanile

Soda Hall Entrance*

Apartment Hall*


* Note: soda hall and apartment images displayed are compressed to fit the 25MB max size for upload utility

Part 2: Recover Homographies

We will need to compute a homography to warp the photos to be able to stitch them together. I chose to do an inverse warp to avoid aliasing problems (using similar interpolation code to proj3). We can fix the bottom right element of H to be 1 (the scale factor) and solve for the rest of the parameters. Although 4 points uniquely identify a homography, I chose to take more than 4 points, resulting in an overconstrained system to reduce the effects of error in point selection. For this reason, I solved for the H matrix using least squares using the following formula*:

* credit to this website for the derivation.

To more accurately choose points, I zoomed in and selected them in Photoshop.

Part 3: Warp the Images

Now that I had a homography matrix H, I could warp the photos. To stitch 2 photos together, I would inverse warp the second one (on the right) to the first one (on the left). For an arbitrary number of photos n, I would finish the process of stitching the first n - 1 photos together, and then stitch that result with the nth photo.

Part 4: Image Rectification

As an aside, we now have the tools to rectify images. That is, given an image of an object (taken at an angle) that we have outside knowledge to be rectangular, we can warp the image to be rectangular.

Painting 1 (Original)

Painting 1 (Rectified)

Painting 2 (Original)

Painting 2 (Rectified)

Part 5: Blend the images into a mosaic

Now that we have the images warped, we want to stitch them together. An easy way to do this is to select the left image to be the shared canvas. The right image will have precomputed left padding since I am doing an inverse warp of points to the left image. I add right padding on the left image to make the two images the same size. Then I create a mask and do laplacian blending (code taken from proj2).

Campanile Mask

Campanile Image 1 (Padded)

Campanile Image 2 (Warped, Padded)

Campanile Panorama


This panorama looks pretty good. You can see a slight seam in the sky since unfortunately the exposure/camera settings seems to have changed between photos.

Soda Mask

Soda Image 1 (Padded)

Soda Image 2 (Warped, Padded)

Soda Panorama


This panorama looks good too. The seam is a little more obvious this time unfortunately due to different exposure/camera settings (sigh "idiot cameras")

Apartment Mask

Apartment Image 1 (Padded)

Apartment Image 2 (Warped, Padded)

Apartment Panorama


This panorama looks good too. Seam is less visible than the other indoor one.

The coolest important thing I learned from this project was accomplishing my homography and warping without for looping over pixels. I forced myself to think in terms of vectorized functions and use numpy functions to efficiently code.