Overview

In this project, I took many overlapping photos, and stitched them together to form one image that contains all the information of all of the other images. In the following sections, I will describe the process which I took to create the Mosaic.

Shooting Pictures

I took multiple pictures of the same scene, each shifted such that they overlap so I can find corresponding points to match.

Selecting Points

Recovering Homography

I defined a matrix H that transforms the point p to p'. Unlike the face morphing, the 3rd coordinate in p' is not exactly 1, since its warped onto a plane that is not parallel to z=1.

[a b c][px]   [wp'x]
[d e f][py] = [wp'y]
[g h 1][ 1]   [ w  ]
  

We solve for w:

w = g*px + h*py + 1
  

We substitute this back into wp'_x and wp'_y:

a*px + b*py + c = g*px*p'x + h*py*p'x + p'x
d*px + e*py + f = g*px*p'y + h*py*p'y + p'y
  

Finally, we can set up two equations like this for every pair of points and form a least squares problem, adding two rows for every pair of corresponding points.

                    [h1]
                    [h2]
                    [h3]
[xi yi 1 0 0 0 -xix'i -yix'i][h4] = [x'i]
[0 0 0 xi yi 1 -xiy'i -yiy'i][h5]   [y'i]
                    [h5]
                    [h6]
                    [h7]
                    [h8]
  

Image Rectification

I took a picture of my bedpost, and a picture of my portable battery and warped them so that the post and battery was facing directly up and at 90 degree angles:

Mosaicing

Here is the result of me putting together the first two images in my settings. There are a couple problems. First, there is a hard boundary between the two images. This will be more evident in the next section. Second, the image is blurry in areas. This is especially evident in the vertical lines on the tapestry.

Weighted Averaging

Here, I do a weighted average. I chose to weight the pixel higher if its closer to its corresponding image's center. That way, theres a gradient from the left image to the right in the pixel weights. This helps with some of the blurriness, and gets rid of the lines between the borders of the images:

Final Results

I took three full panoramas. The first was my apartment's living room. The second is one of my favorite spots on campus. The final panorama is of my friend's rooftop, off campus.

What I've Learned

From this project I ovbviously learned how to calculate homogrophies and blend images together. Beyond that, I found that found that the mosiacs are very sensitive to small changes between images. Even though I did a weighted average to blend the images together, you can see a difference between the leftmost image, and the second from the left. This is because I took the pictures at night, which meant I had to turn the ISO up a lot. the leftmost image was noisy, since there was little ambient light, but the second from the left got the city lights in the background, so it wasn't as noisy. The blending helped with the boundary, but you can still see a difference.

Second, I learned that choosing points is an art. I found that sometimes, selecting more points was not always better. If I selected many points, but in a small area, it would tend to find a homography that satisfied that one area better, sacrificing performance in other areas. By selecting fewer points, but more uniformly spread across the image, I was able to recover a much better homography.