Overview
In this project, we use compute homographies so that we can do projective transforms and then we use warping and blending to construct mosaics. We take two or more photographs and create an image mosaic by registering, projective warping, resampling, and compositing them.
Computing Homographies
After shooting pictures of different scenes from different perspectives, we compute homographies between them, or in the case of rectification, we pick points on a blank canvas so that we can transform into that perspective. To solve for homographies, given pairs of points, p and p', we have equation of the form Ah = b where for each point, we have 2 rows in A of the form [x, y, 1, 0, 0, 0, -xx', -yx'] and [0, 0, 0, x, y, 1, -xy', -yy']. We use least squares to solve for the parameters of h and we obtain our homography matrix, which adds a scaling factor of 1 into the final entry of the matrix.
Image Rectification
Original Laptop
Rectified
Original Monitor
Rectified
Mosaicing
In this part of the project, we take 2 or more photos and warp one into the perspective of the other. Then, we align the photos and blend them together using Laplacian blending as done in project 3. The results are below.
SF Originals
Naive Blend
Mosaic
Crop
Room Originals
Mosaic
Crop
Kitchen Originals
Mosaic
Crop
Conclusion
I learned about how we can use homographies and transformations to create cool panoramas and mosaics using other techniques we've used in the past as well, including warping and blending.
Part 2
Harris Corner Detector
First, we run Harris corner detector to get corner points but there are too many so we run ANMS, in which we follow the description in the paper and make sure that points are only included if they have 90% of the corner strength and has a minimum radius. We take 250 of such points and use them as our corner points.Harris Corner vs ANMS
Feature Descriptor and Matching
In this section, we create feature descriptors to get by taking 40x40 patches around corner points and then we downsample to 8x8 patches and normalize so that we can compare patches across points. We then calculate the nearest neighbor and 2nd nearest neighbor in terms of SSD and set a threshhold to included the point if their 1NN/2NN ratio is below a certain quantity. Below is an image showing the results of feature matching as described here. We can observe some outliers.Feature Matching
RANSAC
In this section, we use RANSAC to refine our points. We randomly sample for a set of 4 points and then compute a homography matrix, H from them. We calculate p' = H* p and then find the SSD between p' and p. Through 10000 iterations we find the homography matrix that best drives down this SSD and grows the largest set of inlier points. Below is a set of matching points after RANSAC.RANSAC
Blending
After this, we blend them together again, and results are here.Results
Manual vs Auto Kitchen
Manual vs Auto Room
