Project 6 : Panoramas Part 2

created by Alvin Wan . Part 1

In this project, we consider another point registration problem, for affine transformations. Here, we compute a homography between (x, y, 1) coordinates. The final 1 value is to account for translations. I begin by labelling points, computing the homography, then warping the image. However, in this part, we automate the point selection process.

1. Select Initial Points.

Below are the steps used in selecting points, for each image.

  1. Select all Harris corners.
  2. Rank all points by their Harris strength and pick the top 25%.
  3. Run adaptive non-maximal suppression per Brown et al.. First all points by strength. Starting from the second-to-largest, compute r--the smallest distance to a corner with larger strength. We then pick the top 200 points, ranked by degree of decreasing radius.

2. Extract Features

We featurize each point by picking (1) a 40x40 patch surrounding it. Then, we apply a (2) Gaussian blur with sigma=1 and downsample by taking every 5th pixel. This gets us an 8x8x3 (d=192) featurization. Finally, (3) we demean and standardize the image to have stddev=1. To visualize, we floored values at -1 and cut off at 1.

3. Match by Features

For each point in one image, we look for the two points in the other image that minimize SSD. If the ratio of these two distances exceeds a certain value (I use 0.67), we include the point.

4. Filter Matches by RANSAC

We repeat the following: (1) Choose 4 points at random from one image, along with its paired point from the other image. (2) Compute a homography. (3) Project all points from one image to the other. (4) Find all projected-point, point pairs where the SSD is less than some threshold (for me, 1000). (5) We call these "inliers" and store them. After 1000 iterations of these steps, we pick the largest set of inliers.

5. Results Comparision with Manual Technique

Finally, construct the homography and stitch the panorama together as before. Below, we start by showing the two original images side-by-side. Below both, we show the combined panorama from (a) to the left and automatically generated panorma from (b) to the right. Note that the latter consistently yields stronger results.

What I learned Interesting that the notion of "corners" could be characterized by eigenvalues, as proxies for "sharpness".

Rectifying Images

This portion was copied from Part 1.

Below, we show some results where the image is warped for a new "perspective". In other words, we show rectified images. First, below is the original image.

From left to right, we flatten perspective facing (1) doe, (2) evans--the building tot he left--, and (3) the ground.