Part A

Louis Leung

A1. Shooting and Digitizing Photos

Here are the pictures we'll be using for rectification/mosaicing. All shot with the same center of projection but rotating the axis slightly.

Deck 1
Deck 2
Louis' Room 1
Louis' Room 2
Miranda's Room 1
Miranda's Room 2

A2. Computing Homographies

We compute homographies by selecting pairs of points (at least 4 pairs) between the two images and solving for Ah=b where h is a vector holding the 8 unknown entries of H. If we pick more than 4 points, we use least squares to find the best homography for the points.

A3. Warping Images

I used inverse warping to get a final image from a warped source image, and just used integer coordinates instead of interp2d to avoid aliasing.

Soda
Soda Rectified
Louis' Room
Louis' Room Rectified

A4. Mosaics

Here we use the previous parts to blend images into a mosaic! I tried both not blending at all (simple binary mask) and alpha blending using a gaussian mask on the binary mask. Looks like not blending at all sometimes works better.

Deck 1
Deck 2
Deck with no blending
Deck with alpha blending
Louis' Room 1
Louis' Room 2
Louis' Room with no blending
Louis' Room with alpha blending
Miranda's Room 1
Miranda's Room 2
Miranda's Room with no blending
Miranda's Room with alpha blending

Part B

B1. Detecting Corners

We use the starter code to detect Harris corners. I set min_distance to 5 since with the default value of 1 the dots were way too concentrated.

Harris corners for room 1 of 2
Harris corners for room 2 of 2

B2. Extracting feature descriptors

I didn't get around to implementing ANMS, so I just hand selected points on the images.

Hand selected correspondences 1 of 2
Hand selected correspondences 2 of 2

Now for each corner, we get the 41x41 pixel square around it, blur and scale it down to and 8x8 patch. We also normalize by taking
(patch - min(patch))/(max(patch - min(patch)))

Feature Descriptor for Point 0 of room 1 of 2
Feature Descriptor for Point 0 of room 2 of 2

B3. Matching Features

For each of our features patches, calculate the SSD to each of the other feature patches. If the ratio of
smallestSSD / secondSmallestSSD
is under a threshhold we set (.3 for examples below), we consider the feature patches to be a match, and pair them up as such.

Matching features that my algorithm detected
Matching features that my algorithm detected

B4.RANSAC

Perform ransac algorithm described here to get our final homography

Correspondences after running RANSAC
Correspondences after running RANSAC

You'll notice that I lost one of my correspondences, since that point ended up not being in the maximal inlier set.

B5.Mosaics

Here's a side by side for the final stitchings. Manual on left, automatic on right.

Louis Room (Manual)
Louis Room (Automatic)
Miranda Room (Manual)
Miranda Room (Automatic)
Deck (Manual)
Deck (Automatic)

Reflections

I think this project as a whole touched on some really cool concepts. My favorite topics were automatic corner detection and creating feature patches. RANSAC is also a pretty cool algorithm.