CS194-26 Project 6B

The the first half of this project's main goal is to explore image mosaicing by warping images taken from different points of view into the same coordinate space. You can find the first part here: Part 6A . In the second part, we'll be using computer methods to determine the points that we use to find our homography matrix. The steps are as follows: Detecting corner features in an image , Extracting a Feature Descriptor for each feature point, Matching these feature descriptors between two images, Use a robust method (RANSAC) to compute a homography, then find the mosaic.

We choose to define interest points as Harris corners, which are based off of corner strength (differences in pixel intensity within a neighborhood). We need to use ANMS to filter out points so we don't have thousands to match upon. Adaptive Non-Maximal Suppression (ANMS), narrows down the points and makes sure they're evenly distributed. ANMS works by assigning a radius to each harris corner, which limits the feature points that exist in each area of the picture. To find the radius, we comparing every interest point to every other interest point in the image. The radius is determined by the minimum distance where that one point's corner strength is significantly more than every other point. By using ANMS, we can narrow this down to 500 points, which we will use to match features between images. Below are the initial corners, then the ones are after is after ANMS refinement.

We need to find features that match between the two images. To do this, we use Muli-Scale Oriented Patches as described in the paper. Essentially, we gather the 40x40 grayscale surrounding each point, then downsample to an 8x8 patch, then normalize with mean and standard deviation. We then reshape to a (1, 64) feature vector.

Now that we've found the features from both images, we need to match them so that the right points are correspondign to each other. To do this, we use nearest neighbors under Lowe's Technique in the paper. For every feature vector, we compute its distance to all the other feature vectors. Once we have the distances, we take the top two closest ones and label them respectively NN1 and NN2. We want to keep the feature vectors such that their NN1/NN2 ratio is less than some threshold, which means that the top two matches are very close. Below are some of the feature matches between images.

RANSAC works by iterating some K times and each time we randomly select 4 points from 1 image to be the inputs for calculating our H matrix. Each time, we choose the Homography which gives us the most inlier points as in features from our feature map match, we select the one with the most inliers by checking the distance. Then we find the best match which gives us the best matches for the points in the inliers.

Making mosaics!

I was really surprised how amazing the automation process made making panoramas and how easy it was to do that. It also made the alignment much better which was really cool. I really enjoyed this part and the results made it worth it. I also learned a lot from feature matching, RANSAC, and then so on.