My Bedroom
by Alfredo Santana
In the first part of this project, I took many pictures and recovered the homography of points of the pictures. Then using the homography, I warped images, rectified images, and blended the images into a mosaic.
For this project I used my iPhone with AE/AF locked to capture the following pictures.
My Bedroom
My Bedroom
Front yard
Front yard
To recover the homographies of my pictures, I selected 8 points in both of my pictures, so that I could construct an overdetermined system of linear equations.
8 selected points
8 selected points
8 selected points
8 selected points
Once I selected all my corresponding points, I constructed a matrix similar to the one below, and I used least squares (np.linalg.lstsq) to solve for the coefficients in the homography matrix. Image from this source.
Homography Matrix
To warp the images, I made the warpImage(im, H) function, which takes the image that will get warped and the homography matrix H. Inside of warpImage I construct a matrix with all the x and y coordinates of the image passed in. I then do matrix multiplication between this newly constructed matrix and H in order to warp the image. I then use cv2.remap to interpolate the result of warpImage.
Outside Warped
Room Warped
Outside Rectified
The most important thing I learned during this project was how to manually compute the homography matrix. It was really interesting learning the math behind it, and it was a great feeling being able to make our own computeH function and not have to use the built-in cv2.findhomography function.
Given the harris.py starter code, I generated Harris Points for both of my images. Below are the Harris Points for my images.
Harris Points Left Bedroom
Harris Points Right Bedroom
Harris Points Left Outside
Harris Points Right Outside
I implemented the ANMS algorithm by iterating over all the Harris points, and for each point where the harris strength is less than 0.9 * H(p2), I store the smallest radius associated with that point. Once I find the suppression radius for all the points, I take the top 500 points with the largest radius as our ANMS points.
ANMS Points Left Bedroom
ANMS Points Right Bedroom
ANMS Points Left Outside
ANMS Points Right Outside
For each of the 500 feature points I obtained from the ANMS algorithm, I crop a 40x40 patch around it and downsample the patch to be an 8x8 patch. I then normalize the descriptor vector by subtracting the mean and dividing by the standard deviation. Below is one of my feature descriptors.
Feature Descriptor Bedroom
After we obtain the feature points and feature descriptors for the pairs of input images, I compute the SSD between all pairs of features descriptors. I get the two smallest SSDs of each feature descriptor and if the ratio of the smallest SSD / secondSmallest is less than a threshold of .1, I keep these feature points. Below are the matching points.
Matching Points Left Bedroom
Matching Points Right Bedroom
After I get all of our matching points, I use the RANSAC algorithm to get our inliner points and use these inliner points to compute a homography. For each iteration of the RANSAC algorithm, I sample 4 random matching points and compute a homography matrix. I use this homography matrix to transform our matching feature points and check if the distance to the matching points in the other image is less than an epsilon of 2. For each iteration, I check how many inliners we kept and after 10,000 iterations we keep the largest set of inliners. After we have our largest set of inliners, I use this to compute a new homography matrix. Below are the inliner points kept after running the RANSAC algorithm.
RANSAC Points Left Bedroom
RANSAC Points Right Bedroom
The coolest thing I learned from this project was how to extract feature descriptors from feature points and how to match feature descriptors of two images.