Image Warping and Mosaicing

CS 194-26: Computational Photography, Fall 2018, Project 6a

Zheng Shi, cs194-26-aad

Project Overview

In this project, we are going to explore the effect of projective transformation, and how we can use it to simulate an image from another perspective (image rectification), and to build a panorama.
Generally, we first define correspondence between two images. Then we can solve for a homography matrix to warp points in image 1 to corresponding points in image 2.

1. Image rectification

To rectify an image, we find 4 (or more) points which are vertices of a regular shape (optimally, a square or a rectangle). Then guess a reasonable list of coordinates which will make sense if we look at it along the normal line of the planar surface. Given these data, a projective warping could "rectify" the image.

2. Mosaic

This time, we need to define a larger canvas to accommodate both images. My approach is to calculate the coordinates of four corners of image 1 after warping, compute the size of new canvas as well as the offset (as coordinates of pixels have to start from 0). And then, simply put both images on the correct position of the canvas and do a blending.

A painting of Brooklyn bridge

Inside St Patrick's Cathedral

Wall of bathroom tiles

3. Summary

1. One tricky part of this project is that you have to keep center of projection unchanged when taking pictures (or, take photos of scenes far away). Otherwise, there will be pretty obvious shadows in the overlapping region.
2. At first, I was going to find a generic alpha mask to blend two images after warping. Such general mask could be hard to generate. An easy way, however, is just to have a vertical border, do a simple alpha blending, and only keep a maximal upright rectangle.
3. Projecting to a plane probably won't work for generating long panorama (like the one in iPhone Camera app), because the images on the side will be stretched a lot and lose all the details.
4. If all parameters of both images are the same, then most likely we don't need to worry about blending even if we use the simplest "taking average in the overlapping region".