CS 194-26 Project 5 [acc id: aez]
Overview
Project 5A: Manual Stitching
Part 1: Shoot The Pictures / Foreword
- Images were shot mostly from the room where I am currently quarantined at.
Part 2: Recover Homographies
- Following this image from lecture, further derivation was done to obtain a nice equation form (derivation attached in code file).
np.linalg.lstsq
was used to solve equation to recover coefficients of H
.
Part 3: Warp The Image
- The
warpImage
function was implemented using inverse warping
- 2 homographies was created –
H_warp_to_original
and H_original_to_warp
H_original_to_warp
was used to find the bounds of the warped images, but passing in the coordinates of the corners of the original image
H_warp_to_original
was used for inverse warping
Part 4: Image Rectification
- Planar surfaces were taken and warped such that they are frontal parallel.
Description |
Original |
Rectified |
Biscuit box |
|
|
Door |
|
|
Part 5: Mosaic
Ouput
- The images look more or less smooth except for the parts where there are a lot of fine details – which is where the images are overlaid. Perhaps it would look better with other reference points.
- The one area to be optimized is how the images are overlayed – curently,
np.max
is used to merge the two photos together
Coolest things learnt
- Probably that the result is highly influenced by the area of correspondence – if the correspondence area picked is highly detailed, the mosaic would likely not look great. I am curious to see if there are any techniques available to mitigate this, especially in manual stitching.
Project 5B: Auto-stitching
Part 1: Detecting Corner Features
Harris corner detector
- Threshold of
min_distance=5
was used in get_harris_corners
starter code to prevent tons of redundant “edges” in sky
ANMS
- ANMS was implemented following the MOPS paper, selecting 500 of the best edges for further processing
- ANMS took quite a long time to run on larger pictures - trade-off between stitching larger images and more accurate edge points for smaller image
Corners
- Images below show each of the 3 sets of mosaics that were done
- Corners are colored neon yellow - please zoom in (especially for ANMS) to see the points clearly!
Harris Corners |
Harris corners with ANMS |
|
|
|
|
|
|
- A 40x40 patch containing the edge point at the center of the patch was sampled, and then evenly sampled to get a 8x8 patch feature descriptor
- Edge padding accounted for at in Part 1
- Descriptors are normalized as specified in the spec
Part 3: Matching the feature descriptors
- Simply done using the quick
dist2
function provided in the starter code
Part 4: RANSAC
- The algorithm was run for 1000 times with a Lowe ratio threshold of 0.4 and inlier threshold of 0.05-0.2
- I included a plot preview of the Lowe ratios to determine what is a good threshold value to eliminate majority of points that might be bad
Part 5: Mosaics / Results
Manual vs. Auto
- I tried auto-stitching the panorama that I did for Part A, and it actually turned out pretty nice!
- A larger picture and less blurry overlap is seen
Part 1 |
Part 2 |
|
|
Other 2 mosaics
Mosaics |
|
|
Coolest things learnt / Futures work
- Understanding and implementing ANMS is probably the coolest part! One especially cool part outlined in the paper is the difference between conceptual implementation and actual implementation - this can probably be carried on to many other fields in computer science.
- In the future, I will probably try to generate edge detection for only a smaller subset of the image since we are assuming that the images would generate a panorama and are in sequence.
- With fewer points, the stitching would be more accurate.
CS 194-26 Project 5 [acc id: aez]
Overview
Project 5A: Manual Stitching
Part 1: Shoot The Pictures / Foreword
Part 2: Recover Homographies
np.linalg.lstsq
was used to solve equation to recover coefficients ofH
.Part 3: Warp The Image
warpImage
function was implemented using inverse warpingH_warp_to_original
andH_original_to_warp
H_original_to_warp
was used to find the bounds of the warped images, but passing in the coordinates of the corners of the original imageH_warp_to_original
was used for inverse warpingPart 4: Image Rectification
Part 5: Mosaic
Input
Ouput
np.max
is used to merge the two photos togetherCoolest things learnt
Project 5B: Auto-stitching
Part 1: Detecting Corner Features
Harris corner detector
min_distance=5
was used inget_harris_corners
starter code to prevent tons of redundant “edges” in skyANMS
Corners
Part 2: Extracting Feature Descriptor
Part 3: Matching the feature descriptors
dist2
function provided in the starter codePart 4: RANSAC
Part 5: Mosaics / Results
Manual vs. Auto
Other 2 mosaics
Coolest things learnt / Futures work