Image Warping and Mosaicing
CS 194-26: Computational Photography, Fall 2018, Project 6a
Zheng Shi, cs194-26-aad
Project Overview
In this project, we are going to explore the effect of projective transformation, and how we can use it to simulate an image from another perspective (image rectification), and to build a panorama.
Generally, we first define correspondence between two images. Then we can solve for a homography matrix to warp points in image 1 to corresponding points in image 2.
1. Image rectification
To rectify an image, we find 4 (or more) points which are vertices of a regular shape (optimally, a square or a rectangle). Then guess a reasonable list of coordinates which will make sense if we look at it along the normal line of the planar surface. Given these data, a projective warping could "rectify" the image.
2. Mosaic
This time, we need to define a larger canvas to accommodate both images. My approach is to calculate the coordinates of four corners of image 1 after warping, compute the size of new canvas as well as the offset (as coordinates of pixels have to start from 0). And then, simply put both images on the correct position of the canvas and do a blending.
A painting of Brooklyn bridge
Inside St Patrick's Cathedral
Wall of bathroom tiles
3. Summary
1. One tricky part of this project is that you have to keep center of projection unchanged when taking pictures (or, take photos of scenes far away). Otherwise, there will be pretty obvious shadows in the overlapping region.
2. At first, I was going to find a generic alpha mask to blend two images after warping. Such general mask could be hard to generate. An easy way, however, is just to have a vertical border, do a simple alpha blending, and only keep a maximal upright rectangle.
3. Projecting to a plane probably won't work for generating long panorama (like the one in iPhone Camera app), because the images on the side will be stretched a lot and lose all the details.
4. If all parameters of both images are the same, then most likely we don't need to worry about blending even if we use the simplest "taking average in the overlapping region".
