Part A: Image Warping and Mosaicing

By Ruochen(Chloe) Liu

Overview

In this assignment I tried image mosaicing. Basically I take several photos and create an image mosaic by registering, projective warping, resampling, and compositing them. Along the way, I have to compute homographies and use them to warp images.

Shoot the Pictures

The first step is to shoot three pictures from the same point of view but with different view directions.

middle

left

right

Recover Homographies

In this part I recover the parameters of the transformation between each pair of images. In our case, the transformation is a homography: p’=Hp, where H is a 3x3 matrix with 8 degrees of freedom (lower right corner is a scaling factor and can be set to 1). Then I transformed it into a least square problem Ax=b and we want to solve for x.Below are the screenshots of the homographies I found for the left and right images respectively.

left homography

right homography

Warp the Images

In this part I warp the images using the homographies I just calculated. Below are the warped images.

Image Rectification

In this part I picked a random image and “rectify” them, which means I warped them so that the planar surfaces are frontal-parallel.

original

Rectified

Blend the images into a mosaic

In this part I blended the warped images into one mosaic. In order to avoid edge artifacts, I used weighted averaging. I firsted padded the middle image so that it has enough space to extend on both sides. Then I warped the left and right images to match the middle one and then blended them together using Laplacian pyramid as I did in the previous project.

Tell us what you've learned

I learned how to apply the knowledge we learned in the lecture to manually stitch images. I learned how to calculate the homogrpahy matrix and how to rectify, warp images with it.

Part B: Feature Matching for Autostitching

Harris Interest Point Detector

In this part I used the provided code to find the harris corners. I noticed that the provided code finds way too many corners so I decided to use corner_peaks instead of peak_local_max to identify corners, which was able to produce much cleaner results.

harris corners

Adaptive Non-Maximal Suppression

In this section, I implemented the Adaptive Non-Maximal Suppression. Interest points are suppressed based on the corner strength, and only those that are a maximum in the neighbourhood are retained.

ANMS

Feature Descriptor extraction

In this part I implemented the Feature Descriptor extraction. Below are some examples of the patches I found.

Feature Matching

In this part I implemented the Feature Matching. Based on the feature vectors of each corners that we calculated in the previous part, we can find pairs of features that look similar and are thus likely to be good matches for calculating the homogrpahy matrix. For thresholding, I used the simpler approach due to Lowe of thresholding on the ratio between the first and the second nearest neighbors.

4-point RANSAC

In this part I implemented the 4-point RANSAC algorithm to compute a robust homography estimate. Below are some mosaics created by both manually stitching and automatically stitching. We can see automatically stitching generally gives better results.

middle

left

right

manual stitching

autostitching

middle

left

right

manual stitching

autostitching

middle

left

right

manual stitching

autostitching

Tell us what you've learned

I learned how to effectively read a research paper and implement the algorithms described in it. I also learned this autostitching algorithms which I found really inspiring and interesting.