Detecting corner features in an image

We first ran the provided Harris corner detector on the input images with paddings of 20 pixels on edges and detected the interest points shown in the first image below. We then implemented the Adaptive Non-maximal Supprission algorithm described in the paper by ranking the Harris corners with their suppression radius (r), which is the distance to the nearest point with a corner strength that is greater than 1/C_r of the current point’s strength. We took the top 250 points (shown in the image on the right) with C_r set to 0.9 for feature extraction.

Extracting a Feature Descriptor for each feature point

For each of the 250 feature points obtained in the previous step, we cropped out a 40x40 patch around it and downsampled it to a 8x8 grayscale patch. We then normalized the values by substracting the mean and dividing the values by the standard deviation and reshaped the patch as vectors to use as feature descriptors. Some of the descriptors of the features from the previous step are illustrated below.

Matching these feature descriptors between two images

We obtained the feature points and descriptors for the pairs of input images. We then computed the Eucliean distances between all pairs of features in the two images. We used a 0.1 threshold on the 1-NN/2-NN ratio as the criteria for accepting the best (nearest) matching for each feature. The matches obtained in this step are shown below.

Use a robust method (RANSAC) to compute a homography

Finally, we used the RANSAC algorithm to compute a stable homography based on the feature matches. For each iteration, we randomly sampled 4 matches and used them to compute the homography matrix. We then apply homography transformation for each matched feature points in one image using the matrix and check if they are close to the matching points in the other image. We count the number of such “inliers” and kept the largest set of such points over 1000 iterations. Finally, we computed the homography matrix with least-squares with all points in the largest set of inliers. The largest set of inlier points obtained from running RANSAC is shown below.

Producing the mosaics

We used the automatically computed hompgraphy matrices for each pair of input images to transform and stitch them together to produce mosaics similar to what we did in part A. The automatically produced results and the manual results from the previous part are shown below.

Tell us what you’ve learned

We learned how to automatically extract and match features in a robust way for recovering the homographies instead of manual matching. Seeing how well the random method worked was really interesting to me.