CS194-26: Image Manipulation and Computational Photography
Sping 2020
Project 5: AutoStitching Photo Mosaics
Zixian Zang, CS194-26-act

Overview

This project is all about stitching together many photographs to create larger composite images.

Part A: IMAGE WARPING and MOSAICING

Section 1: Recover Homographies

The projective transformation can be modelled by the following formula. In order to solve these unknowns, we need four pairs of corresponding points (i=1).

Section 2: Image Rectification

Section 3: Blend the images into a mosaic

After warpping the images, I used a linear blend to combine the images.

Part B: FEATURE MATCHING for AUTOSTITCHING

In this part of the project, I implemented a feature matching algorithm to match and stitch the images together.

Section 1: Harris Interest Point Detector

I started with using Harris Corner Detection algorithm to extract all interest points within the images. Here is a example of the interest points plotted in the images.

Section 2: Adaptive Non-Maximal Suppression

As we can see in the plots above, the number of corners detected is pretty massive. I used Adaptive Non-Maximal Suppression to highlight the local maximum points. The algorithm can be described with the formula below.

The remaining 500 points are plotted below.

Section 3: Feature Descriptor Extraction & Feature Matching

With all the extracted features from images, we need to find features across images that look similar. For the 500 feature points, I computed their pairwise “nearest neighbor” - the least SSD. However, other than taking the match with the least SSD, I considered feature-space outlier rejection:

To control false positive, we filter the matches and only keep the best matches that is greater than a chosen threshold. I used 1-nearest neighbor / 2-nearest neighbor < 0.3 as threshold.

Section 4: RANDOM SAMPLE CONSENSUS (RANSAC)

To control outliers, I further used RANSAC to compute a robust homography estimate. The general procedure is that, I first randomly select 4 features points and compute the homography H. Then I computed inliers where SSD is smaller than the threshold. Repeat 1000 times, and keep the largest set of inliers. This set of inliers is the final set of matches patches that I will proceed to compute the final homography matrix.

Section 5: WARP & BLEND

Then I used the corresponding points to create the automatic mosacing result, similar to the algorithm used in partA.