CS194-26 Project 6A: Image Warping and Mosaicing

In this part we needed pictures to rectify and also create mosaics from. I have images I've taken at the Legion of Honor (taken with Canon T6), my apartment (taken with Sony a6000), a Japanese garden in San Diego and me eating pizza (taken with iPhone), and also screenshots from the videogame Minecraft.

In this part we recover the parameters of the transformation between each pair of images, which are homographies. To compute for the homography, we need to have atleast 4 pairs of points (p) to be transformed to a different 4 pair of points (p') to solve for a 3x3 matrix of 8 unknowns ([a, b, c], [d , e, f], [g, h, 1]) and a value of 1 (H). We will solve for H with the equation p' = Hp. To do this, we will set up a system of linear equations Ah = b where h is an 8x1 vector of the unknown values from H ([a, b, c, d, e, f, g, h]), b is a vector of the x, y coordinates from p', and A is a matrix where for each x,y coordinate pair from p corresponds to a rows of the form [x y 1 0 0 0 −xx′ -yx′] and [0 0 0 x y 1 −xy' −yy']. If there are only 8 equations, then we can solve for h using a regular solver. If there are more than 8 equations then use a least square solver.

Now that we are able to find the homography from set of points p to set of points p', we can warp images to a different plane. We use the warp function from the skimage transform package to achieve this with the inverse map being our recovered homography. In the images below, I warp the ledge from the Japanese garden to a rectange and a pizza box to a square shape.

We can create a mosaic with these images by warping multiple image to the plane of a center image and then stack the warped images on top of them. I padded each image with black borders to allow images space to warp without losing pixel data by going out of bounds and so they can more easily align with the center image. I used weighted averaging and Laplacian stacking (taken from my code in project 3) to blend the images more seamlessly. As you can tell Laplacian stacking is a much better technique than weighted averaging. Each method required the use of masks, which I created on Photoshop.

This project again demonstrated the power of transformation matrices! The biggest technique learned was how to recover homographies, which allowed us to warp images to a different plane and stitch them to another image of similar plane. It's very useful in creating panoramas.