Spencer Hamilton, CS 194, Fall 2021

In this part of the project, Harris Point Detection and Adaptive Non-Maximal Suppression are used to detect interest points in an image which are used as a basis for feature extraction. The features are matched with a nearest neighbor algorithm and used by RANSAC to compute a homography to distort and stitch the images.

Harris Point Detection

Harris Points are extracted from a grayscale version of the image. A pixel cutoff is used to filter corners which could not exist in both images due to angular difference.

Adaptive Non-Maximal Suppression

By selecting Harris Point corners based on their minimum suppression radii, we are able to quickly identify a spacially well-distributed set of 500 interest points.

Feature Descriptor Extraction

Using the filtered interest points, axis-aligned 8x8 patches of pixels are gathered and sampled at low frequency. The vectors are then normalized for bias and gain.

Feature Matching

Each extracted feature in the first image is compared with the features of the second images by finding the Sum of Squared Distance. Using Lowe's method (Bells & Whistles), the ratio of the first and second nearest neighbors are compared (1NN/2NN) to a threshold to find matches.

RANSAC (Random Sampling and Consensus)

As seen in the images, still some of the features matched with Lowe's Method are incorrect. RANSAC is used to further filter the matches by repeatedly randomly selecting four pairs of features and computing a homography. All key-points are transformed with each homography, keeping track of the inliers and outliers of each transformation. The largest set of inliers is kept as our matched features.

Image Stitching and Mosaic

With the automatic feature detection, I am able to create mosaics of images.

What I learned

I learned a lot about the use of statistics in computer vision. I found it extremely interesting that we can build models around error statistics and perform these operations much faster than a human.