Project 6: Auto-Stitching Photo Mosaics
Student Name: Divyansh Agarwal (cs194-26-aba)
Course: CS 194-26 (Computational Photography)
Background
In this project, we use homographies and blending techniques to create image mosaics.
Shoot and Digitize Images
I shot all the images that you will see in this write-up, using my iPhone's camera. For the images used to create the mosaics, I made sure that I only rotated the iPhone along the axis of the camera, and did not translate my iPhone.
Recover Homographies
In order to align our two images together to create a mosaic, I first defined a set of 10-16 correspondences (number of correspondences chosen based on number of clearly visible corners) between the two images of interest. I chose corners to be the correspondences. In order to determine the transformation needed to align the warp the first image to align both images together, I needed to find a matrix H such that p' = Hp, p and p' are coordinates of the correspondences defined, and H is a 3 by 3 matrix with 8 degrees of freedom. I computed H by solving the linear system of equations above by using the approach suggested in this document . In theory, I would have needed to define only 4 pairs of correspondences, but I defined 10-16 correspondences and had an overdetermined system of equations, since with only 4 points the homography would be very unstable and prone to noise.
Warp the Images
Using the computed Homography matrix, I warped one of my images to align its orientation with the orientation of the other image.
I did this by using an inverse warping approach as follows (without loss of generality, let's refer to one of my images as Image 1 and the other as Image 2, with this naming being consistent throughout):
1. I first computed the Homography matrix H that mapped the correspondences between the two images to warp the coordinates of Image 1 to the alignment of Image 2.
2. I applied H to the four corners of Image 1, and got a rectangular bounding box of the warped corners.
3. Within each coordinate of the rectangular bounding box, I applied H^-1 to get the corresponding coordinates from Image 1 to get the pixel from.
4. I used interpolation if the recovered coordinates were decimals and not integers. If the recovered coordinate was not found within Image 1, I would set the pixel value to 0 (this was observed for the cases when I was doing an inverse warp on a coordinate within my bounding box that was not contained within the polygon enclosed by the warped corners)
Rectified images can be seen below:
Original Image
Rectified Image
Another pair of original and rectified images is as follows:
Original Image
Rectified Image
Stitching and Blending
I warped one of my images into the alignment of the other, and then carried out blending to create a mosaic. I found that alpha blending (feathering) gave me good results, i.e., the blended region was computed using alpha * im1 + (1 - alpha) * im2. I implemented a version of alpha blending where I took a blend over the entire overlap region between the two images, and changed the weight given to each image linearly as I moved from left to right. Towards the left of the overlap region, the image correspondiing to the left of the mosaic was given a higher weight, whereas, towards the right of the overlap region, the image corresponding to the right of the mosaic was given a higher weight. Overall, the stitched mosaics looked pretty good!
The first mosaic is as follows:
Image 1
Image 2
Mosaic
The second mosaic is as follows:
Image 1
Image 2
Mosaic
The third mosaic is as follows:
Image 1
Image 2
Mosaic
What I Learned
I gained some useful practice with transformations and warping, as I got to work with homographies. I found this project challenging to code up as I had to deal with lots of bugs that took time to debug, so it was a great way to improve my coding skills as well. The output was super cool and satisyign to perceive! I also learned the importance of defining correspondences well, and appreciate the importance of RANSAC even more.
