Part A: Image Warping and Mosaicing

Introduction

In this project we learned how to create image mosaics, which is commonly known as panoramas. In order to produce this effect, we applied the concept of image homographies, which ultimateley allowed us to warp images and stitch multiple images together.

Recovering Homographies

Homographies play a critical role in aligning the images. The homography transformation matrix H, is a 3x3 matrix. If the scaling factor within the matrix is 1, we will have 8 unknowns. Therefore, in order to solve for H, we would need 4 corresponding points.

Image Rectification

For the following images, we apply transformations on four selected corner points that represent a square. By finding the homographic transformation of the corner points, along with the rest of the image, I am able to warp the images as below.

Original

Original

Original

Rectified (Monitor)

Rectified (Calender)

Rectified (Notebook)

Image Mosaicing

Putting it all together, with two or more images that have a common center point, we can apply the same homographic transformation to produce a panoramic photo. In order to blend the images together, we use alpha blending.

Example 1

1

2

Mosaic

Example 2

1

2

Mosaic

Example 3 (Bad)

1

2

Mosaic

Part B: Automatic Feature Selection

Harris Corner Detection

Harris corner detector is a mathematical operator that detects corners and features within an image. It detects the features by scoring which regions of the image that is most effected when applying small changes.

Adaptive Non-Maximal Suppression

As we can see, there are alot of detected points. In order to reduce the points, we will apply adaptive non-maximal suppression (which was outlined in the paper) in order to spatially distribute 500 points across the image. For every point i, we find the minimum radius to the next point j, where j has a lower percentage of corner strength i.

Feature Descriptors

We converted each 500 point corner points into feature detectors by downsampling the points to 8x8 patches. From there, we can properly use them to find matching points.

Feature Matching

Next, we want to find the features that are most alike. In order to do these we compute the SSD of every feature in the first image with the second image and select the lowest scoring points.

RANSAC

In order to reduce outliers features from above, we apply RANSAC. We do so by select 4 random pairs, compute the homography, and count the the number of inliers. I do this for 5000 iterations, and keep track of the largest inliers. Thus getting the results below:

Results

I display two automatic stiching results and one comparison with with manual stiching



Left: Manual, Right: Automatic


Conclusion

In conclusion, I found this project very intesting. I learned about homographic projections and how it relates to image warping and mosaicing. Based on some of the results above, I can also detect limitations of manually selecting feature points. For Part B, the most interesting part of being able to learn how to detect feature points and effeciently match between two images.