The first step in this project was to actually shoot the pictures. This had to be done in a way such that the transformation between the pictures was projective and there was enough overlap to find common correspondances. I accomplished this by standing in one place with my camera and simply rotating to take multiple pictures. Example photos are shown below, which I used to make a mosaic in a later part.

To recover the homography between two images, we need to find a matrix H that gives us the point(x',y') in the target that corresponds to a point(x,y) in the source image. Specifically, the matrix should do the following:

To find the matrix H, we need find a series of correspondences such that a point(x,y) in the original image corresponds to a point(x',y') in the target image. Once we have the correspondances, we can set up the system of equations below for each correspondance to find the values of H.

Note that we need 4 points to give us a complete set of equations. If we have more than 4 points, we will need to use least squares to solve for the values of H

Once I created a function to calculate the homography transformation between two sets of points, I was ready to start warping images. To do this, I used inverse warping in a manner very similar to Project 3, where I took each pixel of the output projected image, figured out which pixel in the original image it corresponded to using the homography matrix, and used bilinear interpolation to get a final pixel value.

Here are a couple examples of using warping to rectify images. The original images had planar surfaces at an angle, and I took the corners of those planes and transformed them into rectangles in the output image. This allows us to visualize the image from another perspective.

Now that we can warp images into other geometries, we can blend together multiple images into a mosaic. I did this by choosing a center/base image, finding correspondances between that image and the images surrounding it, and finally warping the surrounding images to match the geometry of the base image. Simply stiching together the images after transforming them produced ugly seams, so I used weighted averaging to blend the images together once transformed. The results for three sets of images are shown below

view from my backyard

my street

my living room

I think the coolest thing that I learned is that an image has a lot of data encoded in it, more than what is simply visible to the eye. Image rectification really shows this, as even though the image is only taken from one perspective, it has data on what the image would look like from other perspectives to, which can be recovered using a homography.