In this project, I created a mosaic by taking pictures of various scenes, and selecting corresponding points to define a homography matrix H, which I then used to transform the perspective of one image into another through rectification. The results are then blended in order to create a whole mosaic, and repeated with more images.
Shoot the Pictures
The images were taken with a Google Pixel, and digitized through Matlab.
Recover Homographies And Warp
A homography matrix was created using input points selected using a correspondences tool. The matrix was then used to warp the selected images into a rectified image. Results for the left and right images shown here:
Left Rectified
Right Rectified
Blend Images
The resultant images are then blended into one single mosaic. Currently I'm doing a "brute-force/naive" method, since I might not be able to implement alpha blending in time. But the results are relatively visualized here.
Left Blend
Right Blend
Combined
Summary
I wasn't able to get blending to work in time, but based off of the resulting rectified images, if alpha blending was correctly implemented the resulting mosaics should've looked fine. I spent a long time trying to figure out an issue with the warp, but it turned out it was because I wasn't converting my images to doubles, and getting bright results from that
CS 194-26 Project 6B: Image Warping and Mosaicing
Overview
For this part of the project, I worked to create a more automatic method fo stitching images into a mosaic. This involved muliple steps in order to automatically detect corresponding features.
Harris Corner Detection
To initially identify "corner" features in an image, Harris Corner Detection is used, which identifies all points where there are significant changes in the x and y directions. This identifies a lot of corners, as shown below (over 150,000!).
Adaptive Non-Maximal Suppression (ANMS)
The ANMS algorithm is used in order to reduce the amount of interest points detected with Harris to those that have a strong "corner strength". This is accomplished by calculating the suppression radius for each point detected in Harris, and taking the top 500 points with the largest suppression radii.
Feature Descriptor Extraction
The points of interest are further broken down into descriptors to be used for feature matching. This is done by taking a 40 x 40 area around a point, reducing it to an 8x8 patch that is used to represent the area around the pixel
Feature Matching
Given a specified threshold, we are able to match features to their corresponding image by approximate nearest neighbours. This algorithm helps us determine the set of candidates that would be a potential feature match, and then outliers are dropped in order to reduce the amount of false matches. The image below shows the matching points that our algorithm has reduced our set of 500 points to, leaving just the strongest candidates behind.
RANSAC
I then used 4-point RANSAC in order to compute a homography estimate given the matched points. This is essentially a brute-force m to find the best homography to apply to image based on the matching points lying within the pictures. 4 points are taken at random from the matching points, and a homography estimate is calculated of the given points. If H is a good fit to the rest of the points, then we keep the homography estimate to be used in the image warp.
Mosaic
It seems like the resulting homography matrix calculated from RANSAC does not work as I expected it to. I feel like this may be due to the fact that the way I implemented mosaics in the previous part of the project was a bit crude,
Summary
My implementation from the last part of the project may have affected my overall results. This project was an interesting one, although implementing it for me was extremely tough, which has led to my incomplete results. It was hard to find the right direction to go throughout the project, and even though I was able to test most parts at every step of the way, it was very hard to know how that would've affected the final results.