CS 194 Proj5 Part A + B
Kehan Wang
Part A
Shoot the Pictures
📸 📸 📸
Recover Homographies
To recover the Homography transformation matrix, I take apart the p’ = Hp equation and rearranged it into Ax = b format, of which x is a vector of the 8 degrees of freedom. Then I was able to call np.linalg.lstsq to solve for the 8 unknowns.
Warp the Images
My warping pipeline is as the following:
- compute target shape of the warped image using H and forward homography
- generate a list of all target warping points
- inverse warp target pts to im pts
- filter all out-of-bound points in all the im points
- interpolate to find pixel values of in_im pts
- fill in target from the interpolation values
Example image warp result:
- original image
- target image
- warp result
Image Rectification
By warping towards a rectangle, we are able to rectify a given image.
Example:
-
original image
-
warp result
Blend images into mosaic
Given two images with corresponding points, I can create an empty canvas in the back that is big enough to fit both after blending, and put them in two canvases.
By using laplacian stacks from project 2, I was able to blend these two canvases together with a smooth transition. Here are some of my blending results:
Bells and Whistles:
With the help of our image mosaic blending and a few special “weapons” (a back massage mallet and a pan), I was able to recreate the fight between Kakashi and Obito, two characters in Naruto with 1 single actor - me.
Kakashi and Obito had hated and loved each other, and it was after this fight that all their complications dissolved into friendship. Just like them, I have liked myself and hated myself, and hopefully by morphing into them I can find a way to become my own friend.
Part B
Harris Interest Point Detector
Using corner_harris
and corner_peaks
from scikit-image, I was able to detect pixels with high probability of being corners using a sigma of 1.
Adaptive Non-Maximal Suppression
By first ranking all corners by their H values, then finding the maximum radius of suppression, we can list all corner in order of their radius of suppression. In this way, corners of most significance locally are extracted(I extracted 50 corners in this image) and we reduce the amount of work needed for feature matching.
For feature extraction, I first guassian blurred the picture to reduce aliasing, and then extracted a patch of 40 by 40 with the corner in the middle, but only keep an 8 by 8 image(a downsample by scale of 5). These 8 by 8 patches around my corners are then normalized to 0 mean, 1 std, and flattened to be my feature descriptors.
Examples of my feature descriptors:
Feature Matching
Given two images and their feature descriptors, I used SSD to calculate the similarity scores between two feature descriptors. For each feature in image 1, I calculate SSD for every feature in image 2, and find the most similar and second similar feature. Using Lowe’s threshold, if the SSD of the most similar over SSD of the second similar feature is smaller than 0.1, we’ll consider these two features a good matching.
Images with matching features:
RANSAC - Robust Homography method
Given that we have extracted these matching feature, there maybe one or two features that are not matched and it can completely mess up our homography. Therefore, we use RANSAC to randomly select subsets of matching features, and find the largest set of matching features that consistently fit onto one homography. I used 1000 iterations of RANSAC to calculate the best homography, and used it to warp my images.
Results
- Blend 1
Result:
- Blend 2
Result:
- Blend 3
Result:
Manual vs Auto stitching
Manual stitching introduces error when selecting keypoints. It is also difficult to remember the point sequence in order to create matching features.
- Manual - Auto:
- Manual - Auto:
It is apparent that auto-stitching performs better and saves time and effort.
What have I learned?
I think the coolest thing I have learned in this project is how to read a paper, understand it, and re-implement it.
CS 194 Proj5 Part A + B
Kehan Wang
Part A
Shoot the Pictures
📸 📸 📸
Recover Homographies
To recover the Homography transformation matrix, I take apart the p’ = Hp equation and rearranged it into Ax = b format, of which x is a vector of the 8 degrees of freedom. Then I was able to call np.linalg.lstsq to solve for the 8 unknowns.
Warp the Images
My warping pipeline is as the following:
Example image warp result:
Image Rectification
By warping towards a rectangle, we are able to rectify a given image.
Example:
original image
warp result
Blend images into mosaic
Given two images with corresponding points, I can create an empty canvas in the back that is big enough to fit both after blending, and put them in two canvases.
By using laplacian stacks from project 2, I was able to blend these two canvases together with a smooth transition. Here are some of my blending results:
Blend 1:
Result:
Blend 2:
Result:
Bells and Whistles:
With the help of our image mosaic blending and a few special “weapons” (a back massage mallet and a pan), I was able to recreate the fight between Kakashi and Obito, two characters in Naruto with 1 single actor - me.
Kakashi and Obito had hated and loved each other, and it was after this fight that all their complications dissolved into friendship. Just like them, I have liked myself and hated myself, and hopefully by morphing into them I can find a way to become my own friend.
Result:
Part B
Harris Interest Point Detector
Using
corner_harris
andcorner_peaks
from scikit-image, I was able to detect pixels with high probability of being corners using a sigma of 1.Adaptive Non-Maximal Suppression
By first ranking all corners by their H values, then finding the maximum radius of suppression, we can list all corner in order of their radius of suppression. In this way, corners of most significance locally are extracted(I extracted 50 corners in this image) and we reduce the amount of work needed for feature matching.
Feature Descriptor extraction
For feature extraction, I first guassian blurred the picture to reduce aliasing, and then extracted a patch of 40 by 40 with the corner in the middle, but only keep an 8 by 8 image(a downsample by scale of 5). These 8 by 8 patches around my corners are then normalized to 0 mean, 1 std, and flattened to be my feature descriptors.
Examples of my feature descriptors:
Feature Matching
Given two images and their feature descriptors, I used SSD to calculate the similarity scores between two feature descriptors. For each feature in image 1, I calculate SSD for every feature in image 2, and find the most similar and second similar feature. Using Lowe’s threshold, if the SSD of the most similar over SSD of the second similar feature is smaller than 0.1, we’ll consider these two features a good matching.
Images with matching features:
RANSAC - Robust Homography method
Given that we have extracted these matching feature, there maybe one or two features that are not matched and it can completely mess up our homography. Therefore, we use RANSAC to randomly select subsets of matching features, and find the largest set of matching features that consistently fit onto one homography. I used 1000 iterations of RANSAC to calculate the best homography, and used it to warp my images.
Results
Result:
Result:
Result:
Manual vs Auto stitching
Manual stitching introduces error when selecting keypoints. It is also difficult to remember the point sequence in order to create matching features.
It is apparent that auto-stitching performs better and saves time and effort.
What have I learned?
I think the coolest thing I have learned in this project is how to read a paper, understand it, and re-implement it.