For part 1, I used three pictures to create one mosaic. I found that using three pictures meant that each image had more overlap with its neighboring image, and therefore lead to a bit better result. For each set of images I took, I stood in the same place and only rotated my phone to ensure the images only differed in angle. The first set of images are pictures of my dinner table with a box on it (to add some extra easy to identify corners). The next set of images are pictures of my computer monitor from up close, and the final set of images are pictures of my house. Below are the three sets of input images.
To recover homographies, I first manually selected correspondence points for each image. I used 6 correspondence points for my dinner table, and 8 points for the other two sets of images. Using these points I was able to set up a system of equations that I solved using least squares, to recover the homography matrix H, from the formula p'=Hp. Below are the input points I selected for my dinner table.
With the homography recovered I was able to warp images by the recovered homography. For image rectification, I used two sets of points: one set of points was used to mark the original location of the object to be rectified and another set of points was for the location I wanted that object to go to. I rectified two images: A tile on the floor in my dinner table picture and my computer keyboard in my computer picture. Although not perfect, I was able to get a surprisingly good top down view of both object through image rectification. Below are my results.
To blend the images into a mosaic I warped the second and third images onto the first and stitched them together. I found for the dinner table I got a slightly better result by warping the first and third images to the middle image (this was the only case in which I used this technique). Below are my three resulting mosaics along with their source images.
There is some blurring that occurs (especially in the first mosaic), but I was pleseantly surprised with my results. I believe one issue with the dinner table mosaic was I didn't shoot the picture in the best lighting. I was happiest with the results of my house mosaic which is why I used it as my main example for part 2 (however, there was some inconsitent lighting between the three pictures I shot, which lead to more evident lines in the final image).
The biggest thing I took away from this part of the project was the power of linear algebra. At first glance the task of image rectification seems almost impossible, but through linear algebra (homographies), I was able to create mosaics and rectify images quite well.
To detect corners I used the provided implementation of the Harris Interest Point Detector. This gave me a ton of corners, so I tweaked the parameters to get a more resonable amount of corners. Below are the detected corners in the two images of my house.
I then used ANMS to limit the results to 250 points per image. Below are the results after running ANMS.
Once I had my set of 250 points per image, extracting the feture descriptors was not too difficult. I then compared feature descriptors across the two images to find a matching set. Below are two example extracted features followed by the set of matching features my algorithm found.
Finally, I implemented RANSAC to find a robust set of points for computing my homography matrix. Below are the resulting points after running RANSAC.
After running RANSAC, I had the points I needed to make the final mosaic. I followed the same steps as in part 1 to go from the points outputed from RANSAC to the final product. The results are below. The left image is my result when selecting points by hand and the right image is the autostitching result. Note: when I selected points by hand I used three input images to get a better result, but I only used two input images for autostitching.
Here is an extra autostiched image of a small park in my neighborhod.
I learned a lot from completing this project. Perhaps the most surprising thing to me was seeing just how much information is in a regular image. When I first saw this project would require autostitching, I thought we would have to determine correspondence points through a CNN, similiar to last project, but after reading throgh the paper and implementing the described algorithms I undersatnad why a CNN is not necessary and this method is prefered. I was also surprised to see just how well autostitching worked. Overall, I learned a lot and gained an appreciation for the power of linear algebra.