CS 194 Project 6A: IMAGE WARPING and MOSAICING

Introduction

This project uses homography to warp images transform perspective. Then it blends warped images into a mosaic.

Shoot and digitize pictures

I took pictures from the hallway of an apartment, a prayer closet, and the interior of an apartment. All these locate outside Berkeley campus.

Hallway

Hallway

Hallway

Prayer Closet

Prayer Closet

Prayer Closet

Apartment Interior

Apartment Interior

Apartment Interior

Recover homographies

I did some math on the homography. Eventually I figured to use this following linear system to recover the homographies.
A = [[x1, y1, 1, 0, 0, 0, -x1*x2, -y1*x2], [0, 0, 0, x1, y1, 1, -x2*y2, -y1*y2]], b = [x2, y2], (the actual rows of A and b are more than this because there are at least four pair of correspondent points.) and we use least square solver to recover H. (note: p'=Hp)

Warp the images

I specified the correspondencies and used inverse warping based on H. Here are some images extracted from the prayer closet (see picture below).

Prayer Closet Left Wall

Prayer Closet Right Wall

Prayer Closet Picture

Prayer Closet Drawing

Prayer Closet Letter. The paper is not completely parallel to the wall so the alignment is not perfect.

Blend images into a mosaic

First, I extended all the images into 4000*8000 images, with white area outside the image part. This is for the ease of warping. When I select correspondent points, sometimes one of the pairs have to fall on the white area, which is arbitrary and thus results in discrepancy in the blended image. (see prayer closet example) I used linear blending (set alpha to 1 until edge of images, then gradually decrease to 0, and set alpha to 0 to the part where the image is not shown). I also used Laplacian blending used in project 3. There are many problems with my mosaics, which will be analyzed below.

Source image: Hallway

Source image: Hallway

Source image: Hallway

Hallway mosaic: linear blending. A couple things I did wrong in this mosaic: the first and second input images had two little overlap. My point correspondencies were not sufficient enough either. As a result, the warping of the left side was weird. The alignment was also off among all the three pictures due to lack of point correspondencies. Lastly, linear blending is not very smooth. Between the first and second images, I allowed very narrow transition part, which resulted in sharp discrepancy between the two sides. Between the second and third images, I allowed very wide transition part, which resulted in double images (e.g. the first door on the right wall). There are clearly two different images overlapped with each other.

Source image: Prayer Closet

Source image: Prayer Closet

Source image: Prayer Closet

Prayer Closet mosaic: Laplacian/Gaussian blending (sigma=5 on all images). This time the blending works much better, but there's a small inconsistency between the first and second images in the bottom due to lack of accurate point correspondencies near that area. When I select the points at the bottom, there wasn't enough image area to choose from, so I just select arbitrarily from the white area (the extended area outside the image part, see the beginning of this section).

Source image: Apartment Interior

Source image: Apartment Interior

Source image: Apartment Interior

Apartment Interior mosaic midway: Laplacian/Gaussian blending (sigma=5, border in the middle).

Apartment Interior mosaic: Laplacian/Gaussian blending (sigma=20, border in the middle). This result has inconsistency in color, and we cannot really see the middle image in the final mosaic due to the overlap of bordering (border is always in the middle).

Summary

This is a very cool project. Through many failures, I learned the importance of many details, such as how to take the photos (at peast 40% overlap, etc. I also learned how to speed up the program by combining and simplifying operations. The images are not perfect, but now I know how to make images with better results. :)