CS 194 - Assignment 5A - Stitching Photo Mosaics

Haoyan Huo

Image Warping and Mosaicing

Identifying homography matrices

In this part, we develop the concept of "homography", and use this concept to warp images. An image homography, is defined a 3x3 transformation matrix that transforms points from one coordinate system to another one coordinate system: $p'=Hp$. For example, the identity transformation: $H=\mathbb{I}^{3}$.

To find homographies, one way is to define a set of correspondence points in two different coordinate systems, $\{p'\}$ and $\{p\}$, and the homography matrix $H$ can be found by using least squares: $H'=\arg\min\limits_{H} \sum_{i}||p' - Hp||^2$. Note that the homography matrices only have 8 degrees of freedom: $H=[H_{11}, H_{12}, H_{13}; H_{21}, H_{22}, H_{23}; H_{31}, H_{32}, 1]$.

Once $H$ is found, we can easily transform points $q$ from one coordinate system to another one $q'=Hq$. Usually the last component of $q'$ is not 1, so we need to normalize $q'$ by the last component.

Rectifying surfaces using image warping

To warp image from one coordinate system to another one, I used inverse warping: the set of pixels are defined in the target image, and the coordinates of these pixels are transformed into the original coordinate system. Pixel values of these points are computed using interpolation methods on the original image.

To test this method, I identified some rectangular surfaces in images, and try to find target coordinate systems where pixels are rectangular. Especially, the four corners are transformed into four fixed points: $[100, 100], [400, 100], [400, 400], [100, 400]$.

The following shows another example.

Image mosaicing

The same idea can be applied to mosaic multiple images. The mosaicing procedure can stitch different images of a same object together to create "panoramas". To do this, I selected several points of same objects on different images (as shown in the following image), and warp all images into the coordinate system of the first image.

The resulting mosaic image are created by averaging every pixel of two images. The average is weighted by the alpha value, which is set to 1 if the pixel can be found in the original image, and 0 otherwise.

Another example:

Finally, we show one more example where two images failed to be stitched together. I think the reason is because the points are only chosen around a small area (around the toy panda). In this case, the regions around the panda is stitched well, however the homography matrix has a large variance in regions far from the panda, as suggested by the serious mismatch of the door.