ANMS points: left, middle, right

Feature Descriptors and Matching

We give each point a descriptor. First blur the image by taking a 40x40 box around the center, then blurring things patch with scipy.ndimage.gaussian_filter(patch, sigma=1). Subsample this box every 5 pixels for an (8,8)->(64,) descriptor. Normalize the descriptor with I' = (I - mu) / std to make it invariant to intensity changes.

We use Lowe's technique to match descriptors. Take every feature descriptor in the image we want to warp and find the feature descriptor in the static image with the smallest SSD (1-NN). Then find the second best match (2-NN). We pick this match/pair of features (feature to warp and 1-NN feature) if (1-NN) / (2-NN) < e, where e=.3. In other words, we only keep the best match if the best match SSD is much greater than the next best match's SSD.

Matched points

RANSAC

I used RANdom SAmple Consensus to compute "inliers" to take the final homography. RANSAC chooses 4 out of all the matched points randomly, and computes homography matrix H as in part A. Using H, compute the matched coordinates we want to warp to the perspective of the static image as in part A. Check how close the warped coordinates are to the matching coordinates in the static image. We score each homography H based on whether the SSD between the warped coordinates and those of the static image is < e. The number of inliers is number of coordinates that satisfy this condition. Keep the H with the largest number of inliers. iters=5000, e=2.

Left warp: manual, ransac H1, ransac H2. We notice that automatic mosaic picks different corner points from would I would pick. I focused on the edgees of the pots whereas the math would focus on the ends of the shelf. We get different homographies H using RANSAC, which is why we can have two different warps that can give different perspectives to the panorama. I noticed that RANSAC doesn't work too consistently, as we can get incorrect H since we only pick 4 sample points at a time.

Right warp: manual, ransac H

manual, automatic. We can see how automatically picking features can dissuade ghosting from linear blends.

Leftmost: manual. Rest: automatic

Top: manual. Rest: automatic.

Leftmost: manual. Rest: automatic. I struggled interpolating pixel values between the left and right warp, resulting in the blachground artifacts on the edges. An easy fix is to crop the image.

Leftmost: manual. Rest: automatic. I struggled interpolating pixel values between the left and right warp, resulting in the blachground artifacts on the edges. An easy fix is to crop the image.

Leftmost: manual. Rest: automatic.

Reflection/What I learned

Automatically aligned mosaics were so much better than my manual-defined mosaics. It was interesting to see which points were most likely considered corners because of how the choice could defy my intuition. For example, in the image of handmade clay bowls my friend I made, anms points actually hovered on one side of the shelf, implying that the algorithm might also interpret corners at a higher granularity. It'd be nice to interpolate pixel values more intelligently to perfect a mosaic. It was super cool how some panoramas seemed "larger" or had more depth from changing which H would warp an image, thereby "adding" more perspective into the image.