Project 6-1: Image Warping and Mosaicing

This project taught me many cool features of how panorama images are stitched together, as well as how certain images can have their perspectives changed by using homography.

MANUAL STITCHING

Part 1: Recover Homography Matrix

To recover the homography, I reshaped the matrices and did least squares to find the best fit h vector that transformed points P to points P'. This was the method described in lecture.

Part 2: Warping the Images

I used an inverse warp to remove aliasing by interpolating from the original image. For each pixel position in the warped image, I use the inverse of H to find out where the pixel came from in the original image and interpolated from there. I check that the pixel is in the range of the original image, otherwise I set the value to 0.

Part 3: Image Rectification

For these images, I picked points that I knew were squares or rectangles in the images. And then I artificially create rectangle points simulating the warped points in the desired result. Then I found the homography and warped the image to align by those squares.

Part 4: Mosaic

For these images, I picked points that I knew were squares or rectangles in the images. And then I artificially create rectangle points simulating the warped points in the desired result. Then I found the homography and warped the image to align by those squares.

AUTOMATIC STITCHING

Part 1: Harris Corner Detection

I used the Harris corner detector given to us as a resource, with a minimum distance of 20. I settled on this value after fine tuning all parameters.

Part 2: ANMS (Skipped, unnecessary for good results)

I was still able to get very good results without the use of ANMS. I chose to keep all of the points.

Part 3: Feature Extractor

To extract features, I generated a Gaussian blurred version of each image. Then for each Harris corner, I expanded a box of 40x40 pixels, and read in every 5th pixel, generating an 8x8 patch of the image as the feature for a corner.

Part 4: Feature Matching

To match features, I applied some intuition. First, I only kept corners that were in the right half of the left image, and the left half of the right image. This was to maximize the possibility that I choose points that were actually corresponding by increasing the chance that they are in the overlapped region.

Then, I simply matched them up by applying SSD to each possible pair between points in both images and picking the highest match for each point.

Part 5: RANSAC

I then applied the RANSAC loop 5000 times. Each loop picked 4 random pairs and calculated the exact homography of those 4 pairs. This homography was then applied to all points to see how many were within some error threshold. After 5000 loops, the homography that did the best for the most pairs of points was kept and used, and that largest set of points was used to generate the final homography.

From this point, it was simply applying the exact same algorithm as in the manual stitching. With automatic stitching, we can see that it isn't as accurate as manual stitching. This is most likely because the points picked are not always pixel perfect by the computer. (This can be seen more clearly in the coffee picture)