CS 194-26: Fall 2020
Project 5A: Image Warping and Mosaicing
Megan Lee

Shoot the Photos

In this project, we want to stitch together photos to create a paranoramic style photo. Thus, I shot two photos of my living room from the same point of view but from different view directions, and with overlapping fields of view, such that the transforms between them will be projective.

Recover Homographies

We are able to relate two images with the same camera center by casting a ray through each pixel in the first image and draw the pixel where the ray intersects in the second image. We warp these pixels by applying a homography. Formulas are shown below. I used 8 points (we need to use a minimum of 4, but each additional point makes the homography more accurate), and solved the correspondences with SVD. I initially used least squares, but found SVD to be more effective.

Image Rectification

To verify my warp function is working, let's try rectifying a few images. I tried rectifying a few flat objects that I took from an angle into a frontal-parallel view.

My warp function works pretty well!

Blend the images into a mosaic

Finally, we get to stitch together the mosaic. After I warped my images, I create a final image space with lots of padding so that the mosaic doesn't get cropped out. Then, I added both the first image and the warped image to the blank final image. In order to stitch together the two images, I first tried directly adding them, which did not work out well because it manipulated the colors in the overlapping region. Then, I tried a mask and splicing method, which worked out alright but produced some strange edge artifacts. Finally, I tried taking the maximum of both images, which produced the best results.

The entire process is visualized below.

Original Images

Selected Points

Warped Images (second image warped)

Stitched Mosaic

What I learned

This project was really cool. I found it really interesting that we are able to warp one image into another image's plane, recovering homographies, without knowing any additional information aside from the two images at hand. I was also surprised by how accurate this is. I experience weird warping effects when I try to rectify (and projective warp) some images I've taken on my iPhone, but in this project I was able to almost perfectly rectify flat objects. Since I manually went in and selected the corner points of the object, the warp was near perfect. There's so much from historical images that we're able to recover within seconds, that took some researchers years in the past! Every project in this class makes me appreciate what technology can do for us.