Here are the photos I shot for the project:
|Left Image||Right Image|
First I define the 4 points in each image with the ginput function of matplotlib. Then I construct the A matrix and b vector in order to solve a least-squares problem with the form Ah=b for the unknowns in the homography matrix, where for one point:A=[[x2, y2, 1, 0, 0, 0, -x1x2, -x1y2].
After I get h, I reshape it into a 3 by 3 matrix and the last element is 1. The homography matrix is as follows, where H is the result of the least squares.[[H, H, H],
|Before Rectification||After Rectification|
|Right Image||Pano Image|
I used my phone and borrowed a friend's device to record the flow of the water fountain. On the left are the original videos and on the right is the mosaic. I handheld the devices so the video is not that stabilized.
I use the Harris corner detector to find corners, then apply ANMS to pick 100 points with highest suppression radius which is defined to be the minimum distance from that point to any point with corner strength than it. Then I extract a patch of size 40x40 around each point, apply a Gaussian filter to each patch and normalize it. To find the feature matching, I calculate the ratio of 1-NN squared error with 2-NN squared error, and designate a match if the ratio is less than 0.2 in accordance with the paper. From left to right below are the Harris corners, the ANMS result and the Feature Matching result:
|Harris corners||ANMS result||Feature Matching result|
After getting the feature points, I apply 4-point RANSAC by randomly choosing 4 pairs at a time, calculate the homography matrix, and find the number of correctly mapped points. I do this for 1000 steps and choose the homography that generates the most number of correctly mapped points, for which I apply part A's warping and mosaic to get the following images. The second example is more "straight" while the third example has much less ghosting effect than the result from part A's hand-picked points.