Project 6A - Image Warping and Mosaicing

CS 194-26 Fall 2018

By Tao Ong

In this class project, we take two or more photographs and create an image mosaic by registering, projective warping, resampling, and compositing them.

Recovering Homographies

Before we can warp our images to align with each other, we need to recover the parameters of the transformation between each pair of images. This can be represented with a homography matrix, with is a 3x3 matrix with 8 degrees of freedom. I recovered this homography matrix using (p, p') pairs of points on both images, and finding the matrix that would map them to each other.

Recovering the homography matrix

As seen from the equation above, we have to find the degrees of freedom a-h (since i is known to be 1). I managed to successfully an write equation to calculate this H matrix, which I can now use for image rectification!

Image Rectification

Now that we can calculate homographies, we can warp an image to another image using the inverse of a recovered homography. This technique is particularly useful for image rectification, which involves transforming some object in an image such that it looks like we are viewing it from a different perspective from how the picture was actually taken.

We only need 4 points between two images to calculate a homography, so I tried to find images with rectangular subjects that were taken from an angle. I then plugged the corners of this rectangle and the corners of where I wanted the rectangle to map to into my recoverHomographies function, and applied its inverse in warping the original image. This is the result:

Original

Warped

Original

Warped

Image Mosaicing

Using image rectification, we can create mosaics by including the extra step of blending between two different images.

I approached this by picking one of two images as a base, warping the other image to it using the same image rectification method, then finally blending the results together. The blending was done by simply finding pixels where the two images overlapped, and applying weighting for the pixels from both warped images. This is the result:

Left

Right

Mosaic

Left

Right

Mosaic

Reflection

I thought this project was incredibly cool having used image rectification and image stitching features in mobile apps and Photoshop/Lightroom. I now have a much better understanding of how these features work under the hood!