This project revolves around warping images using homographies. Through homographies, we can change the perspective at which we view an image. This can give us some interesting effects, like making an image shot pointing towards the horizon appear as though we're looking directly at a floor! Mosaics are made by taking several images revolved around the same projection point, warping the images to lie on the same projection plane, then stitching the images together.
I took these pictures on my Nikon D3500 using a 50mm lens, shot at 100 ISO at apertures around f/11 at a shutter speeds of 1/60s. I set the aperture high and made sure to lock the focus between shots so that the images blend as smoothly as possible. I also borrowed a friend's tripod to ensure that all images in each panoramic shared roughly the same center of projection.
Weren't we told as kids not to give our addresses out on the internet?
I recovered the homographies between images using least squares. Each image needed at least 4 correspondences, and I set up about 7-8 for each image.
I followed the following setup for the homography calculation:
Let the following image be represented by the equation, z^ = Hz:
(Source: https://towardsdatascience.com/estimating-a-homography-matrix-522c70ec4b2c)
Suppose we have n data points. Suppose we take the matrix, H, and convert it into a vector x, as shown below. Let the following image be represented by the equation, Ax = b, and let h_33 = 1.
(Adapted from: https://towardsdatascience.com/estimating-a-homography-matrix-522c70ec4b2c)
The system likely has no unique solution, though the matrix A is taller than it is wide. Therefore, we can run least squares to solve for the best value of the vector x, representing the homography H, to fit these points.
Before a homography is applied to an image, we need to account for the need to resize our images. To do so, we apply the homography to the four corners of our image, then pad our image according to how far these points extend beyond the boundaries of our original shape. The homography matrices are also adjusted using the translation matrix (and its inverse) presented in class, based on padding applied to the top and left of the image. Letting T be the matrix that converts our image from padded coordinates to non-padded coordinates, the new homography matrix is calculated as T_inv x H x T.
For my project, image rectification is done by selecting four points in an image that are known to be square, then computing the homography from those four points to a set of four points that are already square. That way, when we apply the homography matrix to our image, the square in the image appears to be parallel to our projection plane.
Each of the mosaics that I created includes three images (shown in the Shoot the Pictures section above). The padding required by each image is calculated at the beginning, then all the paddings are combined into one. The combined padding is then applied to all images. The mosaic is built iteratively, starting with the base image, then warping each image and stitching the warped image to the cumulative mosaic. Stitching is done using Laplacian Pyramid/multiresolution Blending with 2 bands. The mask for each blend is computed as follows: mask(x, y) = 1 if only the cumulative image is present at (x, y), 0.5 if both images are present at (x, y), and 0 if only the newly warped image is present at (x, y).
The original images are shown in the above section, named "Shoot the Pictures"
The coolest thing that I learned from this project is that we can change the perspective of an image through a simple matrix multiplication. It was quite a shock to me that I could make an image look as though it was taken with the camera pointing straight towards the floor. Additionally, it was interesting learning about how we can use more than four key points to create a better estimate of the homography using least squares.
https://inst.eecs.berkeley.edu/~cs194-26/fa21/hw/proj4/
https://towardsdatascience.com/estimating-a-homography-matrix-522c70ec4b2c
https://en.wikipedia.org/wiki/File:Dwinelle_hall_uc_berkeley.jpg
https://www.berkeleyside.org/2020/09/24/uc-berkeley-shut-campus-ban
