CS 194-26 Project 5: [Auto]Stitching Photo Mosaics

Henk Tillman

Part 1: IMAGE WARPING and MOSAICING

Subpart 1: Rectification

After finding an image that I wanted to rectify (am image of Emperor Napoleon III that I took in the Louvre), I selected four points at the corners of the painting. Then I defined four points of a rectangle that I wanted to rectify the painting onto. Using these four points, I could apply least squares in order to solve for the homography matrix H which converted between the two coordinate systems.

Next, I took the four corners of the original image and applied H in order to find the destination coordinates. Using these, I found the size of the target image.

After that, I used skimage.draw.polygon to iteratively list all of the coordinates in the target image. I used the original four points, but this time swapped the order of the coordinates in order to find the inverse homography matrix H_inv. Using this H_inv matrix, I found the point in the original image which corresponded to each point in the target image (oftentimes they did not exist, so I had to filter out the points which were outside of the range of the original image). Finally, I assigned the pixel value at that point in the original image to the corresponding point in the target image. Once all of the points were processed, I had my rectified image!

Before and After

Subpart 2: Image Mosaicing

Once I had the ability to warp images based on homography, I could attempt to combine two images into a larger unified image. I took two images of the San Francisco Bay from my balcony, each only differing by a rotation of about 30 degrees. I specified eight corresponding points in both images on recognizable landmarks: Alcatraz Island, the bay bridge, parts of the SF skyline, etc. Then I computed the homography from the left image to the right (H) and the right to the left (H_inv). I chose to warp the left image into the perspective of the second.

Using H, I warped the corners of the left image into the perspective of the second. Then I found the size of the target image by taking the extremes of the transformed corners and the original right image corners.

In order to populate the target image, I created a polygon using skimage.draw.polygon and the four transformed corners of the left image. I applied H_inv to the coordinates yielded by this function, and assigned the pixel values of all of the original coordinates in the target image to the pixel values at their corresponding coordinates in the left image yielded by H_inv. After this was done, I had finished mapping the left image onto the target canvas

Next, I mapped the right image onto the target image. This was quite simple, as the matching coordinates between them only differed by a constant translation (the translation is due to the fact that the left edge of the right image maps to somewhere in the middle of the target image, as the left edge of the target image needs to map to approximately the left edge of the right image). Since there were conflicting pixels here that were already populated by the left image, I assigned each pixel to the max of the right image pixel and the preexisting pixel value.

The original images

Left image
Right image

The mosaiced image