CS194-26 Project 4A: Image Warping and Mosaicing
Jinyoung Bae
Part 1: Shoot and digitize pictures
First, I used my Samsung Galaxy A6 phone to take pictures, rotating at an angle and trying not to translate the reference point. This is to ensure that
I can reconstruct homographies cleanly.
Jimmy the Snake
Wall Painting
Hallway
Part 2: Recover Homographies
Here, I computed the homographies to warp the photos to stitch them together. I set H to be a 3x3 matrix with 8 degrees of freedom and the bottom right element of H to be 1 (the scale factor).
Using 6 identified points, I solved for the rest of the entries of H using least squares formula. Although only 4 points are needed to solve this system, 6 creates an overconstrained system to reduce errors in point selection.
Below is the system set up with 4 points.
* credit to this website for the derivation.
Part 3: Warp the Images
After finding the homography matrix H from the previous part, I used the matrix H to warp the photos.
Part 4: Image Rectification
After implementing part 3 and being able to find homography matrices, I can warp shapes to line up or become a certain shape.
Here are some examples of making shapes line up so that the lines are corners of shapes are at 90 degrees and it looks like it was taken straigh in front of the object oriented correctly.
Crooked Outlet (Original)
Straightened Outlet (Rectified)
Crooked Ipad on Bed (Original)
Straight Ipad on Bed (Rectified)
Part 5: Blend the images into a mosaic
Now that I can warp images, I stitch the images together by aligning the images using code from Project 2 after warping one of the images to the other image that I am trying to warp to to create a panoramic picture.
After warping the images and aligning them, I blend the two images by creating a mask and doing a 2 layer Laplacian Pyramid.
Jimmy Left View (Padded)
Jimmy Right View (Warped, Padded)
Jimmy Panorama
This panorama came out pretty good. The eyes and computers and snake lines up very well and seems like a very natural image overall.
Painting Left View (Padded)
Painting Right View (Warped, Padded)
Painting Panorama
This panorama was super cool as you can see the writing as well as the art is stitched together perfectly, and even the writing is very natural.
Hall Left View (Padded)
Hall Right View (Warped, Padded)
Hall Panorama
This panorama came out really nicely too as you can see the whole hallway extend all the way down.
The coolest thing I learned from this part was putting together everything we have learned thus far and being able to make
something cool from it. Bringing together concepts like image alignment, masks, and Laplacian pyramids to help us
create our own panoramics was really fascinating and enjoyable!
Project 4B: Autostitching
Harris Interest Point Detector
Using the provided code, we find all the Harris corners in an image at a single-scale level.
Adaptive Non-maximal Suppresion
For each harris point that we found from the previous part, we take the distance to the nearest point where the
Harris corner strength h(p1) < 0.9 * h(p2), and save the radius of this point. Then, after going through all the points,
we take the top 500 points with the largest associated radii, giving us nice spread out interest points.
Feature Descriptor Extraction
After applying ANMS to our Harris corner points, we extract the features from sampling a 40x40
patch around the feature, downsampling it to an 8x8 patch, and then normalizing it. Attached is an example.
Feature Matching with Lowe Ratio
Here, we match various feature descriptors by taking the two nearest neighbor errors using SSD,
and saving the points whos correspondences had a sqaured Lowe ratio (nearest_neighbor_error/2nd_nearest_neighbor_error)^2 < 0.27.
We save the group of points that are chosen from here. The points that are chosen here actually correlate.
Random Sample Consensus (RANSAC)
Following the RANSAC Process:
1. Select 4 random pairs of the Lowe points found in the previous part
2. Compute exact homography on those pairs
3. Find inliers where dist(p', Hp) < epsilon
4. Repeat stpes 1-3 10,000x
5. Keep Largest set of inliers and compute final Homography using those points through Least-squares Homography
Autostitching Mosaics
Using the final Homography matrix found from RANSAC, we stitch our images together as we did in Part A.
Jimmy Hand Stitched
Jimmy Auto Stitched
Painting Hand Stitched
Painting Auto Stitched
Hall Hand Stitched
Hall Auto Stitched
It is interesting to note that for some images, autostitching looks cleaner, while for others, such as the Hall image,
hand stitching produces better results.
What I Learned
I think the feature matching part of this project was really cool. It was very interesting to see
how we can identify features automatically and even correlate between features and find corresponding features between images.