Project 5: [Auto]Stitching Photo Mosaics

Part A

Jeffrey Huo 

For the first part of the project, we create a homography matrix in order to make one part of an image be in the same perspective as another part of an image. By doing this, we can stitch and blend images together in order to create a mosaic. 

Recover Homographies

In order to recover the homography matrix, we find points that correspond to different images. By using these points, we want to solve this system of linear equations.

We solve this using least squares and we are able to create the homography matrix by reshaping the 8 x 1 vector to a 3 x 3 matrix by appending an extra one.

Warp the Images

In order to warp the images, we must find the bounds of the pano image. This can be done by applying the homography transformation to the corners of one image and deriving the size as well as the offset from the results of the transformation. 

Image Rectification

We can perform image rectification by computing a homography between an image and a flat surface. The first result uses the window as the basis.

Blend the images into a mosaic

Here are some results from my basic mosaic creator. First, I find the homography matrix between a left and right image. Using this, I compute the size of the new image and how much this new image is offset from the original using the homography transformation. I then warp the left image, shift the right image by the offset, and finally blend the two images together. Alpha blending is used to blend the two images together, however, I did not spend enough time on the blending and will need to apply a laplace stack to make it look smoother. Here is an example.

Part B

Harris Interest Point Selection

First, we use the provided Harris Corner Detection code to detect corner points in two images.

Adaptive Non-Maximal Suppression

We use ANMS in order to select the best 500 points with the highest suppression radius. This is needed to filter out the best features in an image, due to the many points created from the harris algorithm. 

Feature Extraction and Matching

For each point, we want to extract a feature by using a 40x40 window around the point, downsampling it and using these features to compare across images to find the best corresponding points. We then keep the feature points that satisfy a ratio between the first and second nearest neighbors to reject outlier points. 

Random Sample Consensus

My RANSAC implementation did not end up working but these were the results 

Results

These were the pano results that turned out quite terribly because of the faulty RANSAC implementation. The left is the manual feature implementation and the right is the automatic one.