In this project I rectified images and formed image mosaics by using homographies to align the images together.
Here are all the raw pictures I took for this assignment:
To compute the homography transformation matrix from one set of points into another, I needed to set up a system of equations. Basically, I used equation 13 from here. To expound a bit upon it, we know that our homography should transform our points in the first image to points on the rays passing through the points in the second image and the focal point. However, we can't say where on the ray they'll end up. So we introduce a free depth variable z_a to account for this, but this means that there could be multiple matrices that transform the points onto different depths along the ray. To constrain the equation, we then fix h_33 to be 1. Then by subsituting in for z_a, we can remove it from the equation and instead get it in terms of h_33, which we already predetermined to be 1, as in equation 13. Finally, we can solve the equation using least squares.
To get the actual points for the homography computation, I used matplotlib's ginput. However, to help correct inaccuracies in my annotations, I use the pixel near my annotation with the greatest NCC instead.
To rectify images, I computed the homography of a rectangular object in the picture onto a rectangle on the image plane. To determine the size of the output image I computed the range of the transformed corners of the original image. Then I used inverse warping to fill in the output image.
|
|
|
|
|
|
To blend multiple images together, I would iterate through each image and warp them all into the first image's plane. With each new image, the output mosaic would grow. I had to keep track of how the original image plane was shifting w/ respect to the image indices as a result of expanding the image. To blend the images together, I weighted each pixel's contribution by a ramp/pyramid that had a weight of 1 at the center of the warped image and 0 at the furthest corner, and at the end, I would divide by the sum of the weights across iterations. In addition, I also had to keep track of out of bounds pixels in the output mosaic. To account for this, when I compute sum of weights, I only use the weights at non zero pixel values. After iterating through the images, for pixels with 0 total weight, I instead set the weight to infinity, so that pixel becomes black on the final output.
As a note, I accidentally took one of the pictures in the glade mosaic with a different lighting setting, so it makes the edge stand out