CS194-26 Project 6: Image Warping and Mosaicing

By Kaiwen Zhou

Overview

In this project, we're exploring a form of image warping with an interesting application - image mosaicing. We take several photographs and create an image mosaic by registering, projective warping, resampling, and compositing them. Central to this project is computing homographies and using them to warp images.

Shooting Photos

Unfortunately, I don't have a digital camera or a tripod. Instead, I managed to take a series of vertical images using my iPhone by rotating it while holding it in place. The locations I chose included Memorial Glade, my friend's kitchen, and a dance competition (outside of Berkeley).

Memorial Glade

Prelude

My Friend's Kitchen

Recovering Homographies

In order to rectify and warp images, we first have to recover the transformation parameters between each pair of images. In this case, our transformation is a homography H in the form p’=Hp, where H is a 3x3 matrix with 8 degrees of freedom. One way to recover the homography is via a set of (p’,p) pairs of corresponding points taken from the two images. I used ginput to select point correspondences between images. Because our system is overdetermined, we can't solve directly for the matrix H - I instead solved for the explicit formulas for each variable in H, and set up a least squares equation to solve for the variables themselves.

After some math, the least squares equation amounted to:

Image Rectification

As a preliminary test of our warping function, we perform a simple rectification of an aspect of an image into a rectangular shape. For my rectification examples, I chose my mac keyboard & a photo of the campanile.

Mac Keyboard

original
rectified

Campanile

original
rectified

Final Mosaics

For my results, I performed three different types of blending. The first was simple overlap blending in which I only showed another image in black areas of the original image. I also performed a simple weighted blend in which the overlap consisted of half one image and half the other. Lastly, I performed linear blending by weighing each pixel by the distance from each image center. Overall, linear blending gave the best results.

My campanile/memorial glade image blended the best because much of it consisted of blue sky, with few high frequency objects to blend. The campanile is a bit blurry in linear blending, because the aligning is a little bit off.

Campanile

simple blend
weighted blend
linear blend

Prelude

Instead of working on my project on Saturday night, I instead went to a dance competition. While I was there, I decided I might as well take photos for my panorama. The panoramas didn't blend too well, but that was to be expected - my iPhone camera lighting was automatic & was thrown off by the indoor lighting. In addition, people slightly shifted resulting in artifacts in the blended panorama.

simple blend
weighted blend
linear blend

The kitchen also gave pretty respectable results, as much of the blending lines were on mono-colored objects and there were few details to cause artifacts for the blended images.

Kitchen

simple blend
weighted blend
linear blend

Reflection

It's really cool to see the types of transformations that matrices can perform on images! Being able to warp the perspectives of an image seems like a very powerful tool to post-process images after they've been taken.