Overview

This project utilizes projective transformations to stitch together images taken from the same viewpoint to form a mosaic or panaroma.

Shoot the Pictures

Here are various scenes taken by me from the same viewpoint at 2 different angles.

Recovering Homographies

To recover the homography transformation that will allow the images to be in the same plane, I transformed selected points from each image and set up linear equations for least squares in the form Ax = b. Below is an image that exactly describes how the points p1 and p2 are converted into an A and b matrix then passed into the np.linalg.lstsq function. Since there are only 8 dof, our returned result will have the shape (8,1). To convert this to the transformation matrix that we know and love, I appended a 1 and reshaped the vector into a 3x3 matrix.

Warp the Images

Using a similar process as Project 3 and the RectBivariateSpline function, I used the homography matrix from the previous to set up an inverse warp from the original image to the warped image. One caveat is the new bounding box post-warp. This was harder to determine as previously we had an exact triangle to triangle mapping. To determine the bounding box, I warped the corners of the original image and calculated the max range for rows and columns, making sure to translate the image when populating index values as some were negative and the polygon would ignore negative values. These values were eventually translated back when plugging into the inverse warp function. The results of the warp are below.

Image Rectification

For this portion of the project, I used the warp function from the previous part to test my homographies. I took a picture of a poster and playing card at off angles and used the warp function to create a top-down view of the objects. This was done by selecting points on the corners of the poster and playing card and mapping these points to a flat plane with corners that mimicked their rectangular dimensions. The results are below.

Note: The card itself was blurred in the upper right corner due to focusing and is thus reflected in the top-down image. As the blur is exaggerated due to a lack of information in that area.

Blend the images into a mosaic

To blend the images, I shifted the second image such that the selected feature points were in the same location. To do this, a new image array was created to accomodate the increase in image size due to the shift. I then created an alpha layer using the warped polygon and the secondary image. The warped-polygon-only pixels were set to pixel value 1 while the second image-only pixels were set to 0. The overlapping area is set to 0.5. This layer is then multiplied by the warped image and 1-alpha * second image. This effectively averages the overlapping area and creates a stitched image. Notice that since laplacian stacks were not used, the blending and seams are noticeable. One thing that I noticed was that shrubbery and trees were noticeably harder to align as feature points generaly ignored that area (hard to consistently label those locations). The results of 3 stitching scenes are displayed below.

Overall, the most interesting I learned from is once again the importance of feature selection. The points that were chosen ultimately aligned the objects they were outlining (i.e. buildings, cars, poles). However, objects that were more difficult to align such as trees and shrubbery (which may also have moved due to wind conditions) ultimately remained blurry and unfocused.