Overview

In this project, I used created image mosaics by transforming images into the same perspective and then stitching the images together to form a single panorama.

Image rectification

By selecting the 4 corners of a shape in an image, we can rectify it by computing the homography from the points to a rectangle and then performing the transformation on all of the points in the image.

Image Stitching

By defining correspondences between image, we can then transform images to a base image and stitch them together to from a panorama. The more correspondences there are, the more accurate the homography as more points reduces the error on each particular point. To blend images together more seamlessly, I used a alpha mask.

We can do this with 3 images as well!

We can get some cool effect if we take the 2 pictures at different times.

Summary

Homography are cool! In this project learned how panoramas and that it really is just a perspective transform.

Part 2, Feature matching for autostitching

To automatically get the points that we can use for calculating the homography, we can use the harris corner detector. Using a min_distance of 50, we will only keep the highest response corners in a radius of 50 pixels.

To match points from the first image to the second, we find the nearest neighbors of each point of image 1 to each point of image 2 (using ncc) and only keep the ones which the ratio of the distance to the closest neighbor to the distance to the second closest neighbor is less than 0.6789.

Some of these matches are still incorrect so we use RANSAC to choose a random set of 4 points, compute the homography transformation matrix and see how many other points "agree" with this transformation. We then keep the homography that has the most points in agreement with and use those points to calculate a more accurate homography.

Results:

Unfortunately automatic point selection does not work if the pictures are taken under different lighting conditions as ncc is unable to match the points between the 2 images.

Summary

The coolest thing I learned from this project is how to automatically find point correspondences. However this technique does only work with the same subject under the same lighting conditions. Perhaps with heuristic tweaks this can be applied to work under different lighting conditions or even different subjects (would be useful for face morphing).