CS194-26: Project 6b

FEATURE MATCHING FOR AUTOSTITCHING

Joseph Jiang

Introduction

This project we are creating a method to auto-stitch together photo to create mosaics. In order to properly auto-stitch images together it requires several steps to choose the correct correspondence points.

PART 1: DETECTING CORNER POINTS

In the last part, the "good" correspondence points chosen were all corners. So in light of that observation, we use the given harris corner function to compute all the harris points for a given object. This essentially selects all points that are considered harris corners. We need to filter out all the bad candidiates in the set returned in order to properly stitch together two images (elaborated in next steps)

Harris Corners

PART 2: ADAPTIVE NON-MAXIMAL SUPPRESSION OF CORNER POINTS

It's difficult to work with such a huge number of points returned to us by the harris corner function so to filter out the "bad" points selected, we choose the top 500 best corner points using the Adaptive Non-maximal Supression algorithm. This algorithm selects for corners points that are locally the best corner points. The way the algorithm rates every corner point is based on expanding a radius and seeing which corner stays the best corner when the radius expands. Below is an image with the top 500 best corner points filtered by ANMS.

Recification of Varsity

Original Image

Part 3: FEATURE EXTRACTION

For each point (post anms-filter), I created a 40x40 grayscale window around this point as the center. Then, I compressed this 40x40 frame down to 8x8 by using a gaussian filter and resizing. Each of these frames are then normalized using the mean and the standard deviation. Finally, each of these frames are reshaped into a (1, 64) matrix and inputted all into a matrix that contains information on all the frames, shaped (# of frames, 64). This matrix is used in the next step, feature matching.

PART 4: FEATURE MATCHING

Now we used the provided dist2 function and run the function by inputting the matrix we created in the previous step into both arguments. The matrix outputted from the dist2 function will now contain (distances) how different each point is from another based on the 40x40 window we created around each point. We find the best match and second best match and if the ratio of best match/second best match is less than .4 than it is considered a feature point. We gather all pairs of points that satisfy this ratio and output a dictionary of (im1 points: im2 points)

Feature Matching Points

Feature Points from Image 1

Feature Points from Image 2

PART 5: RANSAC

This final step is to narrow down the feature points down to 4 correspondence points that we will use to construction our homography matrix. This algorithm is essentially a brute force method where we test 4 random points from the feature points and create a homography matrix from those 4 points. We then test this homography matrix by comparing the warped im1 point using this matrix with the actual point in image 2. The smaller the difference between the warped point and the actual point will be our best candidate homography matrix. We brute force this 20000 times to ensure that we choose the best homography matrix. After we get this homography matrix, we compute the moasic using the functions that were defined and computed in the previous part of the project (moasic blend)

Feature Points after Ransac

Ransac Points from Image 1

Ransac Points from Image 2

PART 6: MANUAL VS. AUTOMATIC MOSAIC COMPARISON

The mosaic created from the manually selected points are on the left and the mosaic created from the automatically selected points (computed from harris points, anms, feature matching, and RANSAC) are on the right.

Manually Selected Points

Auto Selected Points

Manually Selected Points

Auto Selected Points

Manually Selected Points

Auto Selected Points