The goal of this assignment was to produce mosaics of images by generating correspondence points among images taken of the same scene from different camera angles, generating the transformation from one image to another, warping the images, and blending the warped images together.
After correspondence points were defined by hand and the transformation computed, I warped the images corresponding to the transformation. For a given image, by defining correspondence points of 4 corners of a rectangular surface, we can rectify images, as shown here. Note the distortion along the edges of the image, which can be explained by the fact that such a view can be produced by looking at the object from the periphery of the field of view.
After the images were warped corresponding to the transformations, we merged the images in various ways, each with their various pros and cons.
This project was cool in that very simple blending techniques such as Gaussian blending work surprisingly well for some mosaics. However, a mask along a vertical line doesn't work always, so alpha blending works a lot better in general.
To stich images automatically, we first used harris corners to detect key points on the image. These corners are as shown here
Then, we limited the number of corners using adaptive non-mamximal suppression. A naive algorithm that loops through all points and computes the minimal suppression radius for each point does well enough for this case.
After we computed the points, we extracted features descriptors from each point from sampling from the 40x40 window around each point. The descriptors in each image were used to produce correspondences between the 2 sets of points. We used the ratio between the first and second nearest neighbors to do so. Finally, we refined the correspondences using RANSAC, randomly sampling 4 correspondences and choosing the largest set of points consistent with the correspondences.
The results are as shown
As you can see, the auto stitched image does just as well as the normally mosaiced image, but autostitching doesn't require input correspondences.
This project was interesting because we had to read and understand a research paper to implement the auto stitching techniques. The techniques, though simple, were quite effective in producing mosaics, and require minimal effort from the user.