CS 194-26 Project 5A: IMAGE WARPING and MOSAICING
Xidong Wu
Overview
In this project, I use several images I took and blend them into a mosaic.
Part 1: Shoot the Pictures
The images I took is a house. The digital camera is SONY 7RII with (1/250, F5.6, ISO 200)
Left Image
|
Right Image
|
Part 2: Recover Homographies
In this part, the function computeH is implemented to calculate 3x3 homography matrix. The formulation from CSE485 course notes at PSU [1] is referenced, which is an alternate representation of the equation p’=Hp. After matrix (seen below ) is constructed, the least squre function w = (A' * A) \ (A' * b);
Part 3: Image Rectification
The left image is rectified. We assume the roof is horizontal and so choose the points on the roof as the basis. In addition, we assume the window is rectangel and the its edge is vertical or horizontal.
Left Image
|
Left Image Rectification
|
Another image is rectified. I choose the four vertex on the magazines as the basis.
original image (Crop)
|
Image Rectification
|
Part 4: Blend the images into a mosaic
three-layer Laplacian pyramid is used.
Left Image
|
Right Image
|
Left Image
|
Part 5: What I learn
This project makes me have a better understanding of homography transformation an its applications. Most importantly, I use many technologies covered in the previous project. It is very comprehensive.
CS 194-26 Project 5B: FEATURE MATCHING for AUTOSTITCHING
Overview
In this part(Proj5B), I will create a system to automatically stitch images into a mosaic.
1) Detecting corner features in an image
After the images are converted into B&W version, we use template code to get features points. Muti-scale is not considered. I still take the images in the Proj5A as an example and its the Harris corners are shown below.
Left Image
|
Right Image
|
Left Image with Harris Feature Point
|
Right Image with Harris Feature Point
|
2) Implement Adaptive Non-Maximal Suppression
From all features points, I choose 500 points with ANMS method and the c_robus is set as 0.9 [2]
Left Image with ANMS points
|
Right Image with ANMS points
|
3) Implement Feature Descriptor extraction and Feature Matching
First create 40x40 patch and subsampling them into 8x8 patch.
For the Feature Matching, I find pairs of features that look similar with SSD.For thresholding, we utilize the Lowe of thresholding on the ratio between the first and the second nearest neighbors and the threshold is set as 0.3[2]
4) RANSAC
4-point RANSAC as described in class is implemented to select the points. And then we use these points to compute a best homography matrix.
Left Image after RANSAC
|
Right Image after RANSAC
|
5) Blending
Use code from PartA to merge two images. Both manually and automatically stitched results are shown below. We could see both of them have a good performance.
Automatical blending result
|
Manual blending result
|
Other examples
1. Supermarket
Left Image
|
Right Image
|
Left Image with Harris Feature Point
|
Right Image with Harris Feature Point
|
Left Image with ANMS points
|
Right Image with ANMS points
|
Left Image after RANSAC
|
Right Image after RANSAC
|
Automatical blending result
|
2. Houses
Left Image
|
Right Image
|
Left Image with Harris Feature Point
|
Right Image with Harris Feature Point
|
Left Image with ANMS points
|
Right Image with ANMS points
|
Left Image after RANSAC
|
Right Image after RANSAC
|
Automatical blending result
|
3. Beijing National Stadium
Left Image
|
Right Image
|
Left Image with Harris Feature Point
|
Right Image with Harris Feature Point
|
Left Image with ANMS points
|
Right Image with ANMS points
|
Left Image after RANSAC
|
Right Image after RANSAC
|
Automatical blending result
|
6) What I learn
The biggest thing I learn from this project is how to read a reaserach paper. Before, I always think it is very difficult but I do it. It is also very magic to combine two images automaticall and it is so smart. It work as good as the manual stiching. It gives me much interest to learn more about computer vision.
References
[1] http://www.cse.psu.edu/~rtc12/CSE486/lecture16_6pp.pdf
[2] Brown M, Szeliski R, Winder S. Multi-image matching using multi-scale oriented patches[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). IEEE, 2005, 1: 510-517.