Part 1: IMAGE WARPING and MOSAICING¶
Rectifying images¶
I computed the homography transform by solving the system of equations below (source for image: http://www.cse.psu.edu/~rtc12/CSE486/lecture16.pdf )
I set H[3,3] equal to 1. The x', y' are points in the image to be warped, and the x, y are images in the picture whose perspective we will keep fixed.
First, I rectified some images just to check that the homography was working. Here are some samples.
Rectifying the poster on the wall:
Rectifying the cover of the book:
Panoramas¶
I selected corresponding points between each pair of images by hand. Then, I created a panorama using similar logic to rectifying images. I kept the left image as a normal perspective, and rectified the right image to match the regions of the left image. I combined them by using a maximum.
Example 1¶
The original images here are taken from this tutorial: https://www.pyimagesearch.com/2018/12/17/image-stitching-with-opencv-and-python/ However, I did not use any other code from there (they used OpenCV for the warping anyways).
These are the corresponding points I chose, overlaid with the original images:
And, the panorama:
Next, I did an incredibly exciting panorama of my apartment's kitchen. Here are the original photos, with keypoints:
And the panorama:
Part 2: FEATURE MATCHING for AUTOSTITCHING¶
First, I ran the given Harris corner detector on the sample image below. Looking at the histogram of Harris response values, I thresholded for only corners with a response greater than 0.5, giving an updated set of corners (also below).
Below are some sample 40x40 patches around a few random corners, showing what they look like after resizing to 8x8.
I implemented feature matching by matching those with the smallest distance1 / distance2, where distance1 is the SSD to the closest descriptor, and distance2 that of the second closest. Then, I implemented RANSAC to get rid of outliers.
Before implementing ANMS, I just chose a random set of corners with a Harris strength value greater than 0.2.
Out of the 138 initial matches, I ran RANSAC for 10000 iterations with a threshold of 0.5. The most number inliers on a single iteration was 8. Then, computing H with those 18 inliers, I got a final inlier count of 12. Those 12 are shown as red crosses in the images below:
I then went back and implemented ANMS. Below is an example, choosing 100 as the numer of corners, and 0.9 as the robustness threshold.
For making the panoramas, I used 1000 as the number of corners. Running RANSAC on those gives the below (with red crosses as the final matched points):
Below are comparisons of the manual and automatic mosaics:
Manual:
Automatic:
Manual:
Automatic:
And some more that I didn't do manually, just automatically:
A very impressive, exciting photo of the floor, with a surprise feature by a sofa:
Besides creating the panoramas from scratch itself which was cool to do, one of the coolest parts was seeing ANMS work - the very simple algorithm to implement gave quick improvements.