CS 194-26 Project 6

Part A: Image Warping and Mosaicing

Regina Xu (aat)


Overview

This project warps images with projective transforms using the homography between point correspondences and blends the warped images into a mosaic.

Recover Homographies

I first captured images with overlapping fields of view by changing the direction of the camera and select 4 point correspondences from the images. Given points p=(x, y) and p'=(x', y'), the homography is: p' = Hp (see left image from lecture below). To solve for H, write as system of equations:

x' = x*h1 + y*h2 + h3 - x'*x*h7 - x'*y*h8
y' = x*h4 + y*h5 + h6 - y'*x*h7 - y'*y*h8

and construct matrices A as a (2n, 8) matrix of coefficients and b as a (8,) vector of points (see right image below). Since Ah = b, solving for h through least squares give us 8 values of H (the homography matrix). Finally, add h9=1 and then reshape H into a 3x3 matrix.

Image Rectification

Using H, we can warp the images such that the planes are frontal-parallel with four points in the source image warped into a square.

Original image with 4 selected points

Warped image

Rectified (cropped)

Mosaics

For this part, the idea is to center an image and warp the other image(s) to the centered image. I first pipe the corners of an image through H to get the warped image boundaries, then create a new array with size equal to the total mosaic shape, and fill this array with the blended warped image and centered image.

For blending, I used alpha masks for the images to blend areas of overlap by taking a sum of the images weighted by their respective alpha masks before putting this result in the final image array.

Results

Image 1

Image 2

Warped image 2

Stitched without blending

After blending

Image 1

Center image 2

Image 3

Images 1 and 2

All 3 images

Image 1

Warped image 1

Image2

The left image below is without blending and right is with linear blending. This one was a little messed up due to the fact that the images, when taken at the time, weren't intended to be projective so the scoreboard is misaligned though the baseball field lines blended more seamlessly.

Takeaways

From this project, I learned about homography and image rectification, and how to combine previous blending techniques to create cool mosaics.