Shoot the Pictures

I shot 2 pictures by the beach, one of them heavily rotated. For this image to fit in the projected image later in this project, I had to add a significant amount of right and bottom padding.

Recover Homographies

I used ginput and a separate Python script to display both of these images and select distinguishing points. I selected 8 points from each image to ensure an overdetermined system when solving for the homography matrix. Then, I wrote a function to take in (p, p') and solve for the homography matrix mapping p to p'. Given the matrix of (x, y) coordinates, I set up a matrix of coefficients corresponding to the homography matrix entries that would operate on each coordinate, and used SVD to get the smallest nonzero homography matrix to solve for the unknown homography matrix entries. Since this came out to a 9-element vector, the last step was to reshape it into a (3, 3) homography matrix.

Warp the Images

Next, I calculated the homography matrix warping the right image to the left image's POV in order to warp it to the same view. I created a grid of all pixel coordinates in the right image, applied the homography matrix transform to get the corresponding coordinates in the target space, and used a map function to transfer each pixel value over. Although the second image was padded to be much bigger, you can see that similar landmarks correspond to similar coordinates between the 2 images.

Image Rectification

I verified my warping function by selecting 4 rectangular points on the right wall of this street image, and making a copy that snapped the deeper 2 points to the same plane as the closer 2, recreating what the selection would look like if the right wall was parallel to the camera. Then, I calculated the homography matrix from the imaginary parallel coordinates to the actual points I selected, and used this to warp the original image to that imaginary point of view. In the result, the right wall successfully shows up parallel to the camera.

Blend the images into a mosaic

I tried using a simple alpha blending method, but there was significant ghosting, so I copied the Laplacian pyramid code from Project 2 and was able to make a mosaic out of the 2 images as below.

Tell us what you've learned

The most interesting thing I learned was that, as in the case of the rectified street picture and even my image after warping, applying a transform to change the point of view of an image can yield a completely natural-looking image from the shifted POV. I had assumed that certain parts on the extremes of the image would appear squished or distended.