Project Overview

Humans have a wide peripheral vision. Often times, the images we capture do not reflect our wide perspective. In this project, we will be stitching together images to create mosaics that mimic our range of sight. These images will be taken from the same standing location, but will differ in that I turn the camera between each image. In order to stitch these images together, we cannot simply use copy and paste. Instead, we must "warp" one of the images to the perspective of the other image. Then, we blend the warped first image and the unwarped second image together using linear feather blending.

Rectifying Images

We will first rectify a single image and warp it so we see the content from a different perspective. We do this by defining corresponding points between key edges and where we would like these edges to go. For example, I have a painting here below:

I would like to view this painting head on. I can do this by finding the pixel locations at the corners of the painting and then finding corresponding points that I'd like these corners to warp to. In this case, I'd like the corners to warp to a nice even rectangle. I then use these corresponding points to solve the Homography matrix, a matrix used to transform corresponding points from one image to another. To solve for it, we can set up a system of equations as shown below.

We can set the variable i to be 1 and are therefore left with 8 unknowns. Thus, in order to solve for this matrix, we need at least 4 corresponding points. I picked the corners of the painting for my corresponding points. Below I show my results once I have solved for this matrix and transformed the painting.

Here I demonstrate the same effect on wanting to see a floor headon:

Mosaic Image Blending

Now I will demonstrate how we can apply the above to creating a panorama. First, we take 2 photographs taken from the same position but of different content (with some overlap) as shown below with the glade.

Now, we want to warp the first image to be in the perspective of the second image. We select corresponding points between the 2 images and then follow the rectify logic above to produce a warp:

Finally, we blend the 2 images together using linear blending:

I repeat the process for 2 pictures of soda and 2 pictures of campus:

All in all this project was really cool! I am most proud of the mosaic I produced above. I found out painfully how important it is to have good points of correspondence and how imprecision in these points is the largest source of error for bad blends / warps. In addition, it's pretty cool what linear algebra can do when applied to images. Lastly, iPhones change lighting dramatically between images so this was also quite annoying to deal with to make the panoramas blend nicely.