Stitching Photo Mosaics

Part 1: Image Warping and Mosaicing

Gautam Mittal

Shooting Scenes

Chemistry Library at UC Berkeley.
The interior walls of the Panoramic Berkeley apartment complex.
Philz Coffee at 1 Front Street, San Francisco, CA.

Recover Homographies

The goal of computing a homography is to solve the equation p' = Hp, where p is a set of points on one image and the homography matrix (H) transforms those points into warped points p' of another image. Since this matrix is a 3x3, there are 9 unknowns. However, rearranging this into a system of equations yields that one of the parameters of H is actually a free parameter which can be set to 1 for convenience. The following system of equations was used with 6-8 correspondences per pair of images in experimental results.

System of equations of solved to find a homography matrix H given a pair of images and their aligned correspondences. In experiments, we made this an overdetermined system by providing more than 4 correspondences (to deal with noise) and then solving with least squares. h33 is a free paramter that can be set to 1 for convenience.

Warp the Images and Image Rectification

Inverse warping with the H matrix was used to generate the new scene geometry, and below are some examples of warping being used to rectify an image such that a planar surface is frontal-parallel.

Left: Philz Coffee at 1 Front St, San Francisco. Right: Rectified image of painting in the original frame.
Left: UC Berkeley Chemistry Library. Right: Rectified image of tile on the ground.

Blend Images Into a Mosaic

Given a sequence of images, I first set the canvas coordinate system to be a translated version of the center image in the sequence. I then warped each image in the sequence and placed it on the canvas. To smoothly blend between images, I computed the intersecting area between a warped image and the rest of the canvas and used the middle line splitting the intersecting area as the axis to blend between the two images. The blend axis allowed for a smooth transition between images since this axis was blurred with a Gaussian filter. Below are results on three different scenes.

Philz Coffee at 1 Front Street, San Francisco, CA.
Interior walls of the Panoramic Berkeley apartment complex.
View of the Chemistry buildings from the entrance to Lewis Hall, UC Berkeley.

Lessons Learned

I learned how to effectively leverage homography to automatically stitch images with manually assigned visual correspondences and that the algorithm for doing this is surprisingly simple! I also learned a lot about the algorithm's sensitivity to change of camera settings or alignment of correspondences across images (it is not very robust to noisy correspondences or images at this point).