Project 5: [Auto]stitching Photo Mosaics
CS 194-26 Fall 2020
November 16, 2020
Project 5A: Image Warping and Mosaicing
Shooting and Digitizing Images
Below are three sets of images I will use throughout this project. Two sets are from a recent trip to Colorado. I took the opportunity of school being online to travel and live in Colorado for a month. In Colorado Springs, we were able to hike so many beautiful trails. One of the highlights of the trip was hiking Pikes Peak, a 13 mile hike up the mountain with a 7000 feet elevation gain. We started at 5am and summitted at 1pm. It was one of the most difficult yet rewarding things I have ever done and my first 14 thousander! The first image is from about 10,000 feet up on Pikes Peak! You can see the clouds. My second image is from the Craggs trail in Colorado Springs and the last image is from my backyard in Austin :)
Recovering homographies and warping images
I used cp select to manually select correspondencies amongst the images. Using these correspondencies I recovered the homographies of the images and warped the first image to the second image. The correspondencies and warping are shown below.
Warping and Padding
I double checked my warping function by ensuring it could rectify frontal images. I warped the original image to square points I chose on the image to see the rectification. The results are below on two images.
Creating a mosaic
Using the results from above, in the creating homography and warping and padding section, I was able to create a blended mosaic. I blended the images by taking the max of each pixel value and stitching the images together in that way. There are still residual seams, but for the most part I think that this blending works well.
My biggest learnings from this project were regarding the homographies and warping steps. Learning how to find a homography from least squares was really interesting. As was the process of recovering the new image coordinates from the homography.
Project 5B: Feature matching for Autostitching
Detecting Corner features of an image
Using Harris Point Detection and the code provided to us, I was able to detect corner points on our image. The results are shown below. I noticed that for some images naturally less corners were detected so I used the min_distance parameter to find an optimal number of points.
I implemented ANMS which helped to make the points more sparse by choosing a certain number of points within a certain minimal radius. The results are shown below.
Extracting Feature Descriptors
I created an 40x40 patch around each ANMS point for each image and then applied a gaussian blur to the resulting patch and then downscaled it to an 8x8 patch. These gave us two arrays of feature descriptors for each point in each image, the first for the left image and the second for the right image. Below I have randomly chosen one feature descriptor to display for each image.
Example feature descriptors
I computed the SSD between each of the filters of the first image and each of the filters in the second image. If the resulting difference was less than some threshold epsilon (which for me was 0.7), I kept the point as a match. The purpose of this step was to find "decent" matches between our ANMS points using the patches in the previous step. Below are our images with the resulting matching points.
I used the RANSAC algorithm to compute the best homography for the image. RANSAC essentially chooses four random points from our previous matched features, computes a homography, and then checks how many points are "inliers", or fall under some threshold which I defined as between 1-5 depending on the images and the output. We do 1000 iterations of RANSAC, and return the largest set of inliers, which we determine to be the points that result in the best homography. Below are the pictures of the RANSAC determined points on each of the images. As you can see they almost match exactly in location in either image. Additionally the warped images are shown below separately before the blending has occured.
With the homography computed I was able to use the same warping and blending function as I did in part A of the projects. The results of the automatic vs. manual mosaics are shown below on three different images. The automatic images are on the left and the manual mosaics are on the right.
Mosaics: Automatic vs. Manual
My biggest takeaway from this project was that feature detection could be done without machine learning. As soon as I heard "automatic alignment" in class, my mind went straight to the previous project. I really admire how this algorithm works. Especially RANSAC and the feature matching with the descriptor- it was so interesting to understand and implement. Also this marks my first time implementing anything I've read in a paper before. It was definitely difficult, but Piazza was incredibly helpful. I realize how important it is to communicate effectively in research so that others are able to replicate your results and use your findings.