Running the Code
numpy and skimage are needed to run the code. Matplotlib is needed if new images are used.
Computing H
In order to rectify and warp images, first one must recover the parameters of the transformation between each pair of images. In this case, the transformation is a homography: p’=Hp, where H is a 3x3 matrix with 8 degrees of freedom (lower right corner is a scaling factor and can be set to one). One way to recover the homography is via a set of (p’,p) pairs of corresponding points taken from the two images.
In order to compute the entries in the matrix H, I set up a linear system of n equations (i.e. a matrix equation of the form Ah=b where h is a vector holding the 8 unknown entries of H) where n is the number of point correspondences between the two images (determined through a simple click and select interface I wrote). Since the system may be over-determined, I use least squares to solve for the eight unknowns in h before appending the scale factor i=1 and reshaping it into a 3x3 matrix.
Warping and Image Rectification
The homography matrix H can be used to forward or inverse warp an input to and output or output to an input image. In this case, I use forward warping. In order to test the homography, I "rectify" or unwarp some images with a set of hand-defined correspondences based on knowledge of the images.
E.g. I know that my laptop base is a rectangle with a 3:2 ratio of width:height, so I pass in hand-defined target image points to match that ratio and warp an input image to those points. I also test warping one image to another using clicked point correspondences from each.
Here I warp one image to another's planar surface using point correspondences. The original images are below.
And here are the warped versions.
Mosaic
Finally, I warp a set of images so they are registered and create an image mosaic. Simple feathering (weighted averaging) is used to blend the images together.
Here are the original images used to create the first mosaic of some flags on a hiking trail, and the left image warped to the right.
And here is the mosaic.
Here are the images used to create my second mosaic, from the top of a hill in Berkeley. The image on the left, warped to the right, is also included.
And here is the mosaic.
What I learned
Linear algebra is powerful. Now I can write my own mosaic-making camera app.