CS194-26: Image Manipulation and Computational Photography

Programming Project #6: Image Warping and Mosaicing

Dennis Lee (cs194-26-acy)

Objective

Using homographies, we change the perspective of already taken images.

Image Rectifying

A homography can be used to transform a tilted image into a head-on version. Just define some points on the image, define the target as a rectangle, and have the transformation match those points up

Memorial Glade	East Asian Library	Source Rectangle	Target Rectangle
Professor Efros's Door	Meme	Failure Case	Squares too small

Image Mosaic

We can also transform one image into the perspective of another. Then we can stitch the images together using some method previously learned to create a mosaic or panorama.

...	...	Beach
...	...	Mount Tam
...	...	A field

TIL

Today I learned it's surprisingly annoying to define correspondences in landscape images, even by hand, since a lot of the features are rounded and getting a perfect selection is challenging. Even the slightest error can cause signficant distortions, unless you define way too many points.

Programming Project #6B: Feature Matching for Autostitching

Objective

Using homographies, we change the perspective of already taken images.

Haaris Corner Detector

This corner detector tries to find areas with two significant lines, which look like corners. It does this by constructing a matrix of image deriatives and comparing the product two eigenvalues (distinct lines), or the determinant of the matrix, with the trace of the matrix, or the sum of the eigenvalues. A matrix where the product is high relative to the sum corresponds to a high corner intensity.

Original Image

Corners

Adaptive Non-Maximal Suppression

Haaris generates a ton of points, but it also generates a score for each one. We want points that matter, but unfortunately most features also tend to be close together.

Maximum scores

ANMS

Feature Descriptor

From each corner, we can extract a feature descriptor - a 40x40 region around the corner, downsampled to 8x8. The blurring helps give the features some robustness.

Selected Features

Corresponding Descriptors

Feature Matching

Using the feature descriptors found above, we can create find matches. Rather than taking the absolute closest matches, we can instead find features in image 1 that are similar to exactly one feature in image 2, or those where the closest match in image 2 is much closer than the second closest match.

RANSAC

While most of the feature matches above are pretty good, there are still some fake matches. However, we have narrowed down the possible features to the point where "most" matches are correct. We can sample four matched points randomly, compute the corresponding homography, and see how well it aligns the rest of the points. This is RANSAC. Each corresponding pair is labeled with its own color.

Results

...	...	Beach
...	...	Mount Tam
...	...	Field
...	...	Lake
...	...	Flowers
...	...	Bridge

Disclaimer - last two pictures were originally one, I just cut them apart and stitched them back together as a test

TIL

Today I learned that finding a feature match is a really hard problem and you need to combine a bunch of methods to find a good match. Even after all that, panoramas are still pretty hard to align properly unless you were specifically taking the pictures for that reason. Most of the pictures here were randomly picked from my phone, and they don't line up well at all. Still, this method works a little better than manual selection, and is much easier on the hands.