Parsa Fereydouni - cs194-26-agy
As we did part A of this project we are making image mosiacs. In this part, unlike the previous part, we don't choose the correspondence points manually. We use corners in the images as features that we match together. To choose the corners we use the Harris Corner Detector. To ensure that we are choosing the strongest featrues, and to match these featurues together we do as follows:
1. Detect features (corners) in an image
2. Extract a feature descriptor for each feature point
3. Match the feature descriptors of the two images
4. Use RANSAC to compute a homography
5. Warp images using the automatically generated homography
6. Blend images and output a mosaic
The last two steps of this procedure are the same as in part A fo the project.
Below you can see the corners detected in each of the images. I used peak_local_max
function provided in skimage to get the peak corner intensities seperated by a minum distance of 10
pixels.
It is clear that still too many corners are being detected. We need to suppress weak corners, and do so in a way so that we still get features that are almost uniformly distributed across the images. ANMS tries to generate a more spread out
(increased variance) distribuation of the corners. In a nutshell, we look for the corners with the largest radii, where radius is defined as the distance from a corner to the nearest corner of higher intensity.
To be able to match features together, we need descriptors for each feature. A descriptor is a patch of 8x8 pixel area surrounding a feature. We can then compare these descriptors by measuring their distance from one another using SSD for example.
To imporve our descriptors, we take a 40x40 patch of the grayscale image surrounding the featrue and then downsample (using gaussian blurring to avoid aliasing) to achieve 8x8 patches. We then bias/gain-normalize the descriptors.
To improve the matching process, we compute ratios for each feature as follows:
ratio = 1-NN/2-NN; where k-NN: kth nearest neighbor
We then take the features and their corresponding match that have a ratio above a certain threshold. The purpose of this is so that we ignore features that we failed to find a decisive match for (if the first nearest neighber and second nearest neighbor are relatively close, we are uncertain which neighbor is the right match).
After matching and thresholding we are left with the following features:
RANSAC is an iterative method to estimate inliners. Using RANSAC, we randomly select 4 points and generated a homography, we then apply the homography and calculate the loss
of this specific homography. Finally, we select the points which resulted in the smallest error according to our loss fucntion.
Below are those points:
As described in part A, we use homography, warping and interpolation to stitch images and blend them.
Cropping the image to get the following:
^not my best one!
comparing image 1:
manual:
auto:
comparing image 2:
manual:
auto:
comparing image 3:
manual:
auto: