CS194-26 Project 6: [Auto]Stitching Photo Mosaics

By: Jia Ze Yu

Date: 2018-11-21

Part A: Image Warping and Mosaicing

This project is all about stitching together many photographs to create larger composite images. To achieve this, I find homographies to map points from the space of one image to the space of another image and transform these images to lie in the same plane and blend them.

Shooting the Pictures

I use my smartphone camera (Oneplus 6T's dual camera setup) to snap photos while rotating the smartphone about an axis of rotation.

Recovering Homographies

I first define corresponding points between the two images I want to stitch together. I use the equation for the transformation p' = Hp to compute the matrix for the homography. I only have to solve for the other eight elements in the matrix because the points on our images are only in 2 dimensions and I can ignore the third row and use the last element of that row as a scaling factor.

---

Warping, Rectifying and Blending the images

For rectification, I pick a bounding box over an object and use the warp function to change the shape of the object.

Part B: Automatic stitching

Harris Corners

To detect corners in the image, Harris Corner Detection algorithm measures a response to a particular function and if I see a varied response to the function over some window, it is likely that there is a corner in that location. I set the minimum distance between the coordinates to be 35 to reduce the number of points.

Adaptive Non-Maximal Suppression

To extract fewer points for cheaper computation cost, ANMS finds the minimum radius to the next feature point such that the corner response for the existing interest point is less than some constant multiplied by the response for the other point. Next, it picks some number of interest points with larger radius to the next significant feature point to retain the points with the strongest responses that are also far away from other strong points.

Extracting Features and Feature Matching

To find the actual patches of features, I look at 40x40 interest points, and downsample those patches to be 8x8 such that the high frequency signal that interferes with the accurate matching of features is removed. Then, I normalize the patches to makethe features invariant to the overall intensity differences and an invariant shift to the distribution of RGB. Then, I match the features by calculating the SSD for each feature with every other feature and compute a ratio of the 1-NN to the 2-NN. This helps me select the matches that have a single good match that satisfy this ratio.

RANSAC

To remove outlier matches, RANSAC chooses 4 random points and compute an exact homography from them. Then, it counts the number of points that are inliers in this homography and keep the set with the largest homography. Ultimately, it computes a new homography based on the largest set of inliers.

Auto-stitched Mosaics

---