CS194-26 Project 6a: Image Warping and Mosaicing
Joey Barreto
Introduction
When a sequence of images is taken about a fixed point--like different views from a tripod--the images may overlap if one is trying to construct a panoramic picture. There will be features shared across the pictures, but they will each look different due to the different viewing angles. It is possible to warp any image into any other's perspective using a projective transformation, also called a homography, provided the two images shared the same point of projection (i.e. the camera didn't translate between shots). Once corresponding feature points are chosen between two images, the image below shows the form of the homography matrix:

The matrix has 8 unknowns, and each pair of points gives us 2 equations, so at least 4 pairs will be needed to determine the system, but preferably more will be given to better stabilize the least-squares solution. Since the camera settings often change automatically to some extent between each shot, the warped image will need to be blended into the image whose perspective we kept in order to hide artifacts.


Rectification
Given an image, one can chose any shape or object viewed in perspective and warp its corners to a head-on view (hence the name, quadrilaterals become rectangles). The matrix is then applied to the rest of the image to rectify it. Below are two examples. The left portal of the Japanese lantern and the 18th calendar day are rectified.


before image

rectified image

before image

rectified image

Warping and Blending
Here three pairs of images taken at different but overlapping viewing angles are combined. In each case, the right image is warped into the left's perspective, and then linear blending is used to hide the edge artifacts. The tree image came out a bit off because it was hard to see the corresponding points on the bark.

left image

right image

panorama

left image

right image

panorama

left image

right image

panorama



Summary
I really enjoy the fact that homographies can undo sheared patterns in paintings that might otherwise be extremely hard to visualize.