CS194-26 Project 4A: Image Warping and Mosaicing

William Wang

Shoot the Pictures

I shot pictures of various views with different perspectives using my phone. I made sure to have an overlapping field of view of about 40-70% for each view.

Original Images

Street

Hall

Laundry Machine

Recover Homographies

To stitch the images together, we first need to warp the two images into the same perspective. We accomplish this by calculating a homography from one of the images to the other. By using the formula below, we can get a 3x3 homography matrix with a scale factor of 1. In the formula below, we have exactly 4 points that uniquely determine the homography, however in order to have a more stable/robust homography, we can use additional points and find the solution that minimizes the norm of Ah - b using least-squares.

Warp the Images

Once I have the homography matrix, I can use it to warp one of the images into the POV of the other. This was done by manually determining the source and destination points of the transform, and then using OpenCV's cv2.remap function to get the appropriate transform.

Image Rectification

With being able to calculate the homography matrix, we can manually determine where certain points should be mapped, and get the overall transform from that. As a result, we are able to "recitify" images, meaning that we can force them to be the dimensions/orientation that we want or expect.

KK Slider 1

Rectified KK SLider 1

KK Slider 2

Rectified KK Slider 2

Blend the images into a mosaic

To create a mosaic, I stitch the warped image with a padded version of the unwarped one - I do this by aligning the images, creating an appropriate mask, and blending them with a Laplacian pyramid of size 2.

Street Left View

Street Right View

Street Panorama


We see that the roof lines up really well, whereas the shadows and rain gutter have some issues. This is most likely due to my imperfect capture of the image. For this error to occur, I must have shifted the perspective from picture to picture.

Hall Left View

Hall Right View

Hall Panorama


Although the warping was good, the stitching caused a lot of the right image to be cut off. However, we still see the desired effect on the door handle, as it has included some of the right image as well.

Laundry Machine Left View

Laundry Machine Right View

Laundry Machine Panorama


This panorama worked out well, but we see that the warp on the right image is a little extreme since we get a fairly different field of vision in the original images.

Takeaways

Overall, I learned a lot about how perspective can be altered! I found that the image rectification was particularly interesting as it allows for us to see views that might not be possible otherwise. For example, the use case of image rectification to visualize the tile floor pattern in a painting was super cool!