Sofie Yang
CS 194-26 - Project 5 Autostitching and Photo Mosaics
Project Overview
This project taught methods of using homographies to project images onto the same plane to rectify images and create mosaics/panoramas. Stitching together images allows for a wider view than a single image can provide and create a seamless image.
Part 1
Photographs
Due to shelter in place, I took pictures while on my run, at the grocery store and in my home on my iPhone XS's main camera.
Recovering homographies
To recover homographies, I generated point correspondences between the image and the transformed image. In the case of rectification, the transformed image was represented by a rectangular shape. I adapted the method from MIT CSAIL as follows:
The homography matrix is formulated as the solution to a least squares solution. Then, I added the constraint of H(2,2) being 1 by adding an identity row to the bottom of the matrix and the zero vector.
Rectification Warping
Girl with Pearl Earring
Villa Casale Roman Mosaic
Mosaics with Blending
Clark Kerr
Portico
Grocery Store
Part 2
I used an adapted implementation of the algorithm described in a href="https://inst.eecs.berkeley.edu/~cs194-26/fa18/Papers/MOPS.pdf ">Multi-Image Matching using Multi-Scale Oriented Patches. I will outline the differences in the next few figures.
Adaptive Non-Maximal Suppression
The Harris corners function returned nearly a million results, so we needed to find a subset that represents strong Harris corners and are evenly distributed. To do so, I iterated through all interest points and found the minimum radius at which there is not another point with a response greater than 1/0.9 times the current point response. Due to the high number of points, I did a preliminary sort on the response values of the points to select the 50000 with the highest responses. The points are ordered from high to low min_radius and I returned the top 500.
Feature Matching and RANSAC
Out of the 500 ANMS coordinates, feature matching with a Lowe score of > 0.6 yielded 153 pairs followed by RANSAC with 100 iterations and a SSD error threshold of 1.5 yielded 19 coordinate pairs. I decided to relax the SSD error threshold because more coordinate pairs makes the homography more stable since it is a least squares operation.
Comparing Manually and Automatically Stitched Images
Clark Kerr
The automatically stitched result is worse than the hand result due to a being a few pixels offset on the x axis. I tried re-running RANSAC a few times to get a different set, but this was the best result. It is probably because the points automatically picked were all in the center of the image and the important alignments were on the edge that overlaps with the other image.
Portico
Grocery
Tell us whats the coolest thing you have learned from this project.
I enjoyed learning about reformulating the homography as a least squares problem and the derivation of the A matrix. It was really cool to understand that an image is just a linear space and that the homography is based on the line of best fit.