Project 4A: Image Warping and Mosaicing

Overview

In the first part of the project, I shoot some pictures under the same point of view, recover the homographies according to the corresponding points, warp (rectify) the images, and finally produce some mosaics.

Shoot the Pictures

For the series of pictures used, I shoot them from the same point of view but with different view directions. Also, I make the overlapping region of the two pictures reasonable (50%). Below is an example. I will show you more in the further sections.

hall1
hall2

Recover Homographies

To perform the transformation between two pairs of images, I need to find out the transformation matrix, namely the homography matrix (H). It is a 3x3 matrix with 8 degrees of freedom (lower right corner is a scaling factor and is set to 1). To recover the matix, I need to define the corresponding points on the two images, and I need at least four points. To reduce the noise, I pick eight corresponding points and solve for the matrix using least squares technique. Below is the linear system I set up to solve for H ((x, y) are the source points and (x_hat, y_hat) are the destination points. h_33 is set to 1).

A @ h = b

Image Rectification

Using the homography, I can rectify images. I draw the corners of the item that I want to rectify and define the corners of the warped shape by hand. Both of the items I want to rectify are rectangular, so I define the final shape to be rectangles. Below is the result.

computer with points
corresponding points
computer rectified
tissue with points
corresponding points
tissue rectified

Blend the images into a mosaic

Using the same technique, I can blend the images into a mosaic. First, I draw the corresponding points on two pairs of images, then I calculate the transformation matrix H. I leave one image still and the other one warped towards it. When it comes to combining the two images, I calculate the overlapping region and do a weighted average on that region (the other region is kept untouched, copying the original color over). I set the weight alpha = 0.8 for the still image. Below is the result.

bank

bank left and corresponding points
bank right and corresponding points
bank right (warped)
bank mosaic
bank mosaic croppped version

target

target left and corresponding points
target right and corresponding points
target right (warped)
target mosaic
target mosaic croppped version

Lewis Hall

Finally, to stitch more (three) pictures together, I first blend two images together, and then I calculate the polygon boundary (mainly width) of the warped image (with color, not black) and shift the two-image mosaic to arrange their locations on the final scene. You can see below about how I do this procedure.

hall left and corresponding points
hall right and corresponding points
hall right (warped)
hall mosaic
hall mosaic croppped version

Full mosaic

hall left and corresponding points
hall (mosaic) right and corresponding points
hall left (warped)
hall mosaic shifted
hall full mosaic
hall full mosaic cropped version

The overlapped region is not blended very well, and in the future, I will use Laplacian pyramid to smooth it.

Tell us what you've learned

The most interesting thing I learned is how to rectify the images to get information! The image stores lots of information, and I just need to use some ways to extract it. Another important thing to remember in this project is that I need to choose the boundary points carefully so that I will not lose too much information from the to-be-warped image.

Reference

Picture of the linear system is adapted here: https://towardsdatascience.com/estimating-a-homography-matrix-522c70ec4b2c

The other pirctures are shot by me! They are compressed due to the limit of upload file size.