Project 6: Image Warping and Mosaicing

Author:

Jesse Gao, cs194-26-afi

Part A

Project Overview

Creating image warps and mosaics.

Shoot the Pictures

Ideally the photos taken should be taken with the same camera with a stable axis of rotation. The following were pictures of my apartment room captured by my phone on a swivel stand.

images/left.jpgimages/right.jpg

Recover Homographies

In the next part of the project, we label the point correspondences in the images and compute the transformation.

Image Warping

Now that we have the homography, we can transform an image so that its corresponding points match some other set of corresponding points. I used numpy’s linear algebra functions to apply inverse transformations for the warped image pixels.

Floor art

Tiles[]

Mosaic

Using corresponding points, we can shift an image to “match” another and create a combination of both.

Original Images

images/left.jpgimages/right.jpg

Apartment Living Room

Hand Cropped Version

Lessons Learned

In this project, I learned about the mathematics behind how to compute a homography, how to use that homography to warp an image. Overall it was a very challenging project because of all the small errors that could happen when computing the inverse transform from the original image’s pixels to the target image’s pixels.

Part B

Project Overview

Auto detect interest points and automatically compute the best matching interest points for an Image warp

Finding Interest Points

We start with finding Harris points and then filter them using Adaptive Non-Maximal Suppression. The result is a set points that are of high interest and not all bunched together. Here is a comparison of non filtered vs filtered points.

images/left.jpgimages/right.jpg

Feature Descriptors and Matching

Each interest point has a feature descriptor, which is just the surrounding pixels. We compare these descriptors sets for two images to determine possible sets of corresponding points. These are shown in the plot below with red lines connecting possible corresponding points.

(autoimages/correspondingright.jpg)

RANSAC

Now that we have possible sets of corresponding points, we need to choose 4 that fits the best. We do this by randomly choosing 4 sets and computing the homography and transforming all other points for one of the pictures. We then check if the transformed points match up to the points of the other picture. Once we find the set with the most matches, we use its homography to warp the picture and create a mosaic. Sometimes it turns out ok but it produces really bad results most of the time. It really confused me because the corresponding points RANSAC chose were pretty good most of the time but my manual warps were way better most of the time. I gave up after about 7 hours of messing with tuning variables. I think its because my pictures were too big.

Backyard

Corresponding points

RANSAC points

Generated mosaics manual vs auto

Living Room

Corresponding points

RANSAC points

Generated mosaics manual vs auto

Front Yard

Corresponding points

RANSAC points

Generated mosaics manual vs auto

What I learned

I learned about the application of RANSAC and how to detect interest points in an image. I also learned how matching of interest points work. I think that was the most interesting part of this project. The rest of the project just taught me patience.