CS 194:-26 Image Manipulation and Computational Photography, Fall 2018

Project 6: Image Warping and Mosaicing

Riku Miyao, CS194-26-acq

Part 1: Introduction

The goal of this assignment was to produce mosaics of images by generating correspondence points among images taken of the same scene from different camera angles, generating the transformation from one image to another, warping the images, and blending the warped images together.

Part 2: Warped Images

After correspondence points were defined by hand and the transformation computed, I warped the images corresponding to the transformation. For a given image, by defining correspondence points of 4 corners of a rectangular surface, we can rectify images, as shown here. Note the distortion along the edges of the image, which can be explained by the fact that such a view can be produced by looking at the object from the periphery of the field of view.

Prerectified image
Post Rectified Image
Prerectified image
Post Rectified Image

Part 3:Generating the Mosaics

After the images were warped corresponding to the transformations, we merged the images in various ways, each with their various pros and cons.

Input Image 1
Input Image 2
Input Image 3
Input Image 1 and 2 stiched together using alpha blending. Note the shadowing along the intersection of the 2 images, probably due to parallax.
Image stitched together using gaussian blending along vertical borders. This works surprisingly well for the left mosaic, but has a visible seam on the right side.
Input Image 1
Input Image 2
Image stitched together using gaussian blending along vertical borders. This has a visible seam along the edge of blending.
Image stitched together using linear alpha blending. This works well, though the lighting difference between the two input images need to be compenstated for
Input Image 1
Input Image 2
Image stitched together using alpha blending. This doesn't blend well due to errors in correspondence generation/camera translation.

Part 4 Conclusion

This project was cool in that very simple blending techniques such as Gaussian blending work surprisingly well for some mosaics. However, a mask along a vertical line doesn't work always, so alpha blending works a lot better in general.

Part 5:Automatically stiching Images

To stich images automatically, we first used harris corners to detect key points on the image. These corners are as shown here

Key Points in Image 1
Key Points in Image 2

Then, we limited the number of corners using adaptive non-mamximal suppression. A naive algorithm that loops through all points and computes the minimal suppression radius for each point does well enough for this case.

Suppressed Key Points in Image 1
Suppressed Key Points in Image 2

After we computed the points, we extracted features descriptors from each point from sampling from the 40x40 window around each point. The descriptors in each image were used to produce correspondences between the 2 sets of points. We used the ratio between the first and second nearest neighbors to do so. Finally, we refined the correspondences using RANSAC, randomly sampling 4 correspondences and choosing the largest set of points consistent with the correspondences.

Selected Key Points in Image 1
Selected Key Points in Image 2

The results are as shown

Normal stitched image 1
Auto stitched image 1
Normal stitched image 2
Auto stitched image 2
Normal stitched image 3
Auto stitched image 3

As you can see, the auto stitched image does just as well as the normally mosaiced image, but autostitching doesn't require input correspondences.

Part 6 Conclusion

This project was interesting because we had to read and understand a research paper to implement the auto stitching techniques. The techniques, though simple, were quite effective in producing mosaics, and require minimal effort from the user.