Project 05

Autostitching Photo Mosaics

Devesh Agarwal

Part A: Image Warping and Mosaicing

Shooting and Digitalizing Pictures

For this project I decided to shoot and digitalize my own pictures. I used my smartphone camera after learning how to use exposure and focus locking. Further, it was important to have at least 40-70% overlap for the registration and matching. Due to COVID-19, I only had access to the indoors of my house but I tried to get a variety of types of images.  

Recovering Homography

I created the function *homography* to calculate alignment parameters that I can feed into a transformation function. To solve the linear system, I first tried n = 4. However, as speculated, this was very unstable/ prone to noise. Then I used an overdetermined system with 8 points. I used np.linalg.lstsq in order to match the patches surrounding the points selected on ginput in my get_coordinates function.  

Warping the Images

I wrote the function warped which is a higher order function that uses the homography function to calculate the alignment parameters and then uses the homography to transform the image. You can see this in action in the following images:  

Creating a Mosaic

I used two functions to generate the mosaics. The first is a basic linear blending function, and the second is a laplacian blend similar to my implementation from Project 2.  

Part A Takeaways

Part A of this project included a lot of photography techniques that I was unfamiliar with so the process of going and getting my own images while being cognizant of the camera settings was new to me. Furthermore, I learnt and was exposed to homography and understood how mosaics or panoramas are built in general which was fascinating.  

Part B: Feature Matching and Autostitching

Detecting Corner Features

The first step in autostitching is to detect harris corners inside of an image. This produces a large set of points that we can use to potentially align the two images on. Below you can see the harris corners on my original images:  

Applying Adaptive Non-Maximal Suppression (ANMS)

Directly searching for harris corners provided an images with a large set of corners. Here, I implement an ANMS algorithm in order to select a subset of these corners with a minimum distance between each pair of corners. This provides us with distinct corners that don't have any overlap within their radii and will be appropriate for feature mapping.  

Extracting Feature Descriptors and Feature Matching

I further select a subset of optimal points for feature matching. Here, we are matching the two images to see whether they have matching counterparts in the other image. This is using feature descriptors which use 40x40 patches scaled down to 8x8. Below you can see the small subset of matched features overlaid on each image.  

Implementing RANSAC

Using the methods we learnt about in class, I implemented RANSAC to compute an optimal homography that is not influenced by outliers. In order to do this, I set up an experiment which generated 700 sample choices of 4 coordinates, computing inliers for the samples (within epsilon threshold), and computing the optimal homography by using least squares approximation.  

Part B Takeaways

In Part B, I learnt how to automate the process so that no manual stitching is required in the process. I was really spellbounded by the harris point detection algorithm and the possibilities that come from the ability to match features found as such in images. The mosaics I created are just a started in my journey of understanding how image manipulation can be achieved using just information encoded in the images itself.