Overview
If we use a camera take a picture of the same scene in the same position from a different angle, we can estimate the transformation between the two images as a perspective projection. Determining this transformation matrix H will allow us to merge the images into a single mosaiced image. I shot several images at different angles. These images are shown below.
|
|
|
|
|
|
Recover Homographies
Similar to project 3, in order to get a good result, correspondences (keypoints) must be defined first. To find the transform matrix H, we can unroll the matrix into a column vector h. From here, we can expand the relationship H*v = v', or [[a b c] [d e f] [g h 1]] * [[x0] [y0] [1]] = w * [[x1] [y1] [1]] where w is some arbitrary scaling vector. Expanding this out in terms of H's contents will allow us to find two systems of equations for the variables inside of H (one for each coordinate, x and y). Thus, given four coordinate pairs we can uniquely determine all contents of h. We write this as Ah = b, where A is an 8x8 matrix and b is the an 8x1 matrix consisting of the four points' column vectors stacked on top of each other.
A perspective projection transform is uniquely determined by four pairwise corresponding points, but this would be highly susceptible to noise. To protect against noise, we can overdetermine the system by selecting more than four points. We can then use least squares to determine the closest fitting h, by simply doing inv(A^T * A) * A^T * b.
Warp the Images
Provided the perspective transform, we can warp images to match each other. I used an inverse warp; for each pixel inside my final image, I multiplied the coordinate by the inverse of the homography matrix to determine which pixel in the original image the current value should correspond to, and interpolated if the pixel index was off an integer value. A calculator is shown at two different angles below; one angle is then warped to match the other.
|
|
|
|
|
|
|
|
Blend Images Into a Mosaic
Given two identically warped images, we can overlay the matching parts to create an extended version of the original images. I did this by creating a blank canvas, and adding images onto it. I applied a mask that was 1 everywhere except for where image 1 and image 2 were both non-zero, where I blended the overlapping area together to create the final image. The products of this method are shown below.
|
|
|
|
|
|
|
|
|
|
|
|
What I've Learned
I've learned that transform matrices give us a lot of information! (not just perspective projections). Once a transform matrix is determined we can do a lot of things with it that are visually very interesting.