For this project, we implemented the projective transform, which requires at least 4 pairs of correspondences. Having eight points across two images, we are able to synthetically warp images to a common plane, even if they were taken at different angles. The important thing is that we keep the same center of projection.
The math behind the projective transform is as follows. We first solve for eight variables, a-h. These eight variables represent the degrees of freedom in the 3x3 projective matrix, which describes how two planes relate to one another. Though there are 9 elements, the last one--i--can be set to 1. We solve for these eight variables simply with the least squares approximation for a linear system.
Afterwards we have to warp every image to the plane of the base image. To do so, I used the inverse mapping, which warped from destination to source to avoid splatting.
To stitch the images together into a mosaic, I Laplacian blended the outputs so pixels at the edges of the images were weighted averages of the combined images. To eliminate artifacts, I extrapolated values from the edges of each image.
The coolest thing that I have learned about this project is that planes can be related to each other by matrices--even nonlinear ones. I was very excited to see that by changing the coordinates space that we operate in we could also mosaic and stitch images in cylindrical or spherical space.
Another thing I learned was the importance of not translating the camera in between shots. Many potential panoramas were ruined by the shift in center of projection. I will confirm this in pt. 2, stay tuned. t_t