CS 194-26 Project 5A: IMAGE WARPING and MOSAICING

Xidong Wu

Overview

In this project, I use several images I took and blend them into a mosaic.

Part 1: Shoot the Pictures

The images I took is a house. The digital camera is SONY 7RII with (1/250, F5.6, ISO 200)

Left Image

Right Image

Part 2: Recover Homographies

In this part, the function computeH is implemented to calculate 3x3 homography matrix. The formulation from CSE485 course notes at PSU [1] is referenced, which is an alternate representation of the equation p’=Hp. After matrix (seen below ) is constructed, the least squre function w = (A' * A) \ (A' * b);


Part 3: Image Rectification

The left image is rectified. We assume the roof is horizontal and so choose the points on the roof as the basis. In addition, we assume the window is rectangel and the its edge is vertical or horizontal.

Left Image

Left Image Rectification

Another image is rectified. I choose the four vertex on the magazines as the basis.

original image (Crop)

Image Rectification

Part 4: Blend the images into a mosaic

three-layer Laplacian pyramid is used.

Left Image

Right Image

Left Image

Part 5: What I learn

This project makes me have a better understanding of homography transformation an its applications. Most importantly, I use many technologies covered in the previous project. It is very comprehensive.

CS 194-26 Project 5B: FEATURE MATCHING for AUTOSTITCHING

Overview

In this part(Proj5B), I will create a system to automatically stitch images into a mosaic.

1) Detecting corner features in an image

After the images are converted into B&W version, we use template code to get features points. Muti-scale is not considered. I still take the images in the Proj5A as an example and its the Harris corners are shown below.

Left Image

Right Image

Left Image with Harris Feature Point

Right Image with Harris Feature Point

2) Implement Adaptive Non-Maximal Suppression

From all features points, I choose 500 points with ANMS method and the c_robus is set as 0.9 [2]

Left Image with ANMS points

Right Image with ANMS points

3) Implement Feature Descriptor extraction and Feature Matching

First create 40x40 patch and subsampling them into 8x8 patch.

For the Feature Matching, I find pairs of features that look similar with SSD.For thresholding, we utilize the Lowe of thresholding on the ratio between the first and the second nearest neighbors and the threshold is set as 0.3[2]

4) RANSAC

4-point RANSAC as described in class is implemented to select the points. And then we use these points to compute a best homography matrix.

Left Image after RANSAC

Right Image after RANSAC

5) Blending

Use code from PartA to merge two images. Both manually and automatically stitched results are shown below. We could see both of them have a good performance.

Automatical blending result

Manual blending result

Other examples

1. Supermarket

Left Image

Right Image

Left Image with Harris Feature Point

Right Image with Harris Feature Point

Left Image with ANMS points

Right Image with ANMS points

Left Image after RANSAC

Right Image after RANSAC

Automatical blending result

2. Houses

Left Image

Right Image

Left Image with Harris Feature Point

Right Image with Harris Feature Point

Left Image with ANMS points

Right Image with ANMS points

Left Image after RANSAC

Right Image after RANSAC

Automatical blending result

3. Beijing National Stadium

Left Image

Right Image

Left Image with Harris Feature Point

Right Image with Harris Feature Point

Left Image with ANMS points

Right Image with ANMS points

Left Image after RANSAC

Right Image after RANSAC

Automatical blending result

6) What I learn

The biggest thing I learn from this project is how to read a reaserach paper. Before, I always think it is very difficult but I do it. It is also very magic to combine two images automaticall and it is so smart. It work as good as the manual stiching. It gives me much interest to learn more about computer vision.

References

[1] http://www.cse.psu.edu/~rtc12/CSE486/lecture16_6pp.pdf [2] Brown M, Szeliski R, Winder S. Multi-image matching using multi-scale oriented patches[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). IEEE, 2005, 1: 510-517.