CS194-26 Project 4A: Image Warping and Mosaicing

Jinyoung Bae

Part 1: Shoot and digitize pictures

First, I used my Samsung Galaxy A6 phone to take pictures, rotating at an angle and trying not to translate the reference point. This is to ensure that I can reconstruct homographies cleanly.

Jimmy the Snake

Wall Painting

Hallway

Part 2: Recover Homographies

Here, I computed the homographies to warp the photos to stitch them together. I set H to be a 3x3 matrix with 8 degrees of freedom and the bottom right element of H to be 1 (the scale factor). Using 6 identified points, I solved for the rest of the entries of H using least squares formula. Although only 4 points are needed to solve this system, 6 creates an overconstrained system to reduce errors in point selection. Below is the system set up with 4 points.

* credit to this website for the derivation.

Part 3: Warp the Images

After finding the homography matrix H from the previous part, I used the matrix H to warp the photos.

Part 4: Image Rectification

After implementing part 3 and being able to find homography matrices, I can warp shapes to line up or become a certain shape. Here are some examples of making shapes line up so that the lines are corners of shapes are at 90 degrees and it looks like it was taken straigh in front of the object oriented correctly.

Crooked Outlet (Original)

Straightened Outlet (Rectified)

Crooked Ipad on Bed (Original)

Straight Ipad on Bed (Rectified)

Part 5: Blend the images into a mosaic

Now that I can warp images, I stitch the images together by aligning the images using code from Project 2 after warping one of the images to the other image that I am trying to warp to to create a panoramic picture. After warping the images and aligning them, I blend the two images by creating a mask and doing a 2 layer Laplacian Pyramid.

Jimmy Left View (Padded)

Jimmy Right View (Warped, Padded)

Jimmy Panorama


This panorama came out pretty good. The eyes and computers and snake lines up very well and seems like a very natural image overall.

Painting Left View (Padded)

Painting Right View (Warped, Padded)

Painting Panorama


This panorama was super cool as you can see the writing as well as the art is stitched together perfectly, and even the writing is very natural.

Hall Left View (Padded)

Hall Right View (Warped, Padded)

Hall Panorama


This panorama came out really nicely too as you can see the whole hallway extend all the way down.

The coolest thing I learned from this part was putting together everything we have learned thus far and being able to make something cool from it. Bringing together concepts like image alignment, masks, and Laplacian pyramids to help us create our own panoramics was really fascinating and enjoyable!

Project 4B: Autostitching

Harris Interest Point Detector

Using the provided code, we find all the Harris corners in an image at a single-scale level.

Adaptive Non-maximal Suppresion

For each harris point that we found from the previous part, we take the distance to the nearest point where the Harris corner strength h(p1) < 0.9 * h(p2), and save the radius of this point. Then, after going through all the points, we take the top 500 points with the largest associated radii, giving us nice spread out interest points.

Feature Descriptor Extraction

After applying ANMS to our Harris corner points, we extract the features from sampling a 40x40 patch around the feature, downsampling it to an 8x8 patch, and then normalizing it. Attached is an example.

Feature Matching with Lowe Ratio

Here, we match various feature descriptors by taking the two nearest neighbor errors using SSD, and saving the points whos correspondences had a sqaured Lowe ratio (nearest_neighbor_error/2nd_nearest_neighbor_error)^2 < 0.27. We save the group of points that are chosen from here. The points that are chosen here actually correlate.

Random Sample Consensus (RANSAC)

Following the RANSAC Process: 1. Select 4 random pairs of the Lowe points found in the previous part 2. Compute exact homography on those pairs 3. Find inliers where dist(p', Hp) < epsilon 4. Repeat stpes 1-3 10,000x 5. Keep Largest set of inliers and compute final Homography using those points through Least-squares Homography

Autostitching Mosaics

Using the final Homography matrix found from RANSAC, we stitch our images together as we did in Part A.

Jimmy Hand Stitched

Jimmy Auto Stitched

Painting Hand Stitched

Painting Auto Stitched

Hall Hand Stitched

Hall Auto Stitched

It is interesting to note that for some images, autostitching looks cleaner, while for others, such as the Hall image, hand stitching produces better results.

What I Learned

I think the feature matching part of this project was really cool. It was very interesting to see how we can identify features automatically and even correlate between features and find corresponding features between images.