CS 194-26 PROJECT 5B
Kenan Jiang

Overview
In this project, I am doing image mosaicing.

Shoot the Pictures
In this part, I took pictures from my living from. I shoot from the same point of view but with different view directions.

Recover Homographies & Warp the Images
First I selected some points from the below image. Those points (p) would form a rectangular after warpping. Second, I defined p' myself to a rectangular. Then I compute the H, in the p' = Hp equation, using numpy.linalg.lstsq.

After having H, I did the inverse warpping using the original image and H. This gives me the image rectification of the original image. See below

Blend the images into a mosaic
After checking my H matrix is working, I work a similar process like rectification that warps one image to another. The difference is that instead of mapping to a hand-defined square, I selected coresponding points on the target image. The I added one image_width to each coordinate of target image. I did this because I want to map two images to a large (2 times the size) canvas. After I did this, I can map the warpped picture on one half and the another unwarpped image on another half of the canvas. Here are results.

What I learned from project 5a
I think the most exciting skill I learned is producing a image rectification. I used to thought producing a different picture with details from another angle must requires extra information. However, this homography matrix really opens my mind. Turns out, we just need to rearrange pixels in a perticular way, we can discover so many details from one single-angle image.

Part B: Detecting corner features in an image
I used the given code to find harris corners.

Part B: Implement Adaptive Non-Maximal Suppression
To select valuable points from all the harris corners, I implemented Adaptive Non-Maximal Suppression. Based on the paper and h matrix calculated from harris corners, I first find, for each point A in corners, a set of points from harris corners whose h value times c_robust is greater than A's h value. Then I picked the point with min distance to A. Now each corners has a value indicating the distance to its closest neighbor. The I rank corners by their value and get top 500 points whose neighers are far from them. These points are more indicative as matching pointers later. You can see we have less points.

Part B: Feature Descriptor extraction and Feature Matching
The I took a 41 by 41 window around each corner because I want our corner to be at the center. I downsize these 41*41 window to 8*8, normalize it. Now I have some 8*8 patches for each points.
Now I need to find some coresponding between points from two images. I used the given dist2 function to calculate the distance between each pair of points from two images. If the a point have a signification lower distance with another point compared to all other points, I consider them to be a matching.

Part B: RANSAC and Mosaic Results
I randomly select 4 points and compute H from it. Then I test this H on all points. I put this process in for around 100 times and picked the H that can map the most number of points correctly. Then I use this H to do warpping.

Part B: What I learned in Part B
Reproducing the code from a publication is difficult. To test one's understanding of a paper, the best way might just be recreate the code.