Project 6: Image Warping and Mosaicing Part 1

CS 194-26: Image Manipulation and Computational Photography, Fall 2018
Jonathan Fung, CS194-26-abs

In this project, we attempted to rectify images and mosaic images together by using projective warping. These homographies, as they are called, have 8 degrees of freedom. After choosing a few correspondance points by hand from the original photo to the desired perspective, we solve for the homography matrix, which allows us to warp all points from the original photo to the new perspective.

In order to rectify an image, I selected four points which form the corner of whatever I was trying to rectify into a rectangle. I then formed a rectangle with the other set of correspondance points, then transform the whole image.

In order to mosaic images together, we first define a handfull of correspondance points between the two images. We then solve for the homography matrix using least squares, and then warp one image into the perspective of another. Finally, we overlay the two images to create a mosaic'd image.


Part 1. Image Rectification

For rectification, we select 4 points that are the corners of a rectangle plane, then map them into a rectangle. Note that after we solve for the homography matrix H, we have to take the inverse because we use inverse warping, cycling through the pixels of the destination and applying the inverse transform H_inv. Here are a few results.

A Paper.
A Poster.
A computer.
Paper is now a rectangle, but the words on the hair gel are distorted.
Easier to see the poster now!
Now we can see what's going on!
Windows.
Rectified the lowest window.

Part 2. Mosaics

Now that we know our warping and homographies work, we move on to mosaic'ing together two images. We first take one image, warp it to the perspective of the second photo, and then blend the two together. I tried 3 methods of blending, 1) Overlapping, 2) Maximal regions, Linear Blending. Overlapping entailed overwriting existing pixels. Maximal regions entailed taking the maximum intensity of the pixels that overlapped, and linear blending involved scaling an alpha value at 0 towards the edge of the overlap and assigning the pixel value a weighted sum.

Left side of a room.
Right side of a room.
Left side of a room warped to perspective of right.
Merged with the overlap technique. 6 points of correspondances were used, leading to minor imperfections (chair leg)
Merged with the overlap technique. 12 points of correspondances were used, a much better result.
Merged with the maximal value technique (between overlaping regions)
Merged with mean-based alpha blending linear technique. Chair legs now look perfect, but the window has imperfections.
Merged with min-based alpha blending linear technique. Chair legs now look perfect, but the window has imperfections.




Left side of a pic of the Campanile by Bechtel.
Right side of a pic of the Campanile by Bechtel.
Left side of the scene warped to perspective of right.
Merged with the overlap technique
Merged with the maximal value technique
Merged with the linear blending technique




Left side of a Classroom, with me!
Right side of a Classroom, also with me!
Left side of the classroom warped to perspective of right.
Merged with the overlap technique. Border on the left side.
Merged with the maximal value technique. Lighting was different between the two, so this method was suboptimal, when it performed well in previous pictures.
Merged with the linear blending technique. Performed the best, because of different lightings.

Part 3. Summary

I learned a lot about mosaicing in this photo. Points must be picked nicely, and least squares is a very powerful tool that allows us to measure as many datapoints as we want and fit nicely to all of them!