Harish Palani (CS 194-26)
For this project, I used the following three perspectives captured at a local park here in Portland, Oregon. These shots all revolve around a basketball hoop, which forms a key focal point to be used when aligning images for stitching.
I chose to blend the center and right perspectives, zeroing in on three objects present in each perspective: the basketball hoop, the lamppost, and a street sign in the background. The final points selected for alignment can be seen below.
With correspondences defined by hand as shown above, I had to recover the parameters of the transformation between the two images to properly inform subsequent warping & rectification steps. The resulting homography matrix H is shown below.
Warping and rectification algorithms for this part were tested on the right perspective displayed above, with the warped output ultimately blended wtih the center perspective in the final mosaic. Results for this section are shown below.
To form a realistic mosaic from these two perspectives, I kept the center perspective unchanged and blended it with the warped version of right perspective as shown above. This yields the following raw output — not bad!
Cropping yields the following final output, with minimal artifacts at first glance. Upon closer inspection, the misaligned windows and lamppost in the background — along the seam of the two perspectives — reveal themselves to be perhaps the most glaring errors present.
What you've learned (so far): This has been an incredibly interesting project with many practical lessons. Perhaps the coolest of all has been developing an general understanding of homographies and mosaicing, both of which lie at the core of the panorama feature used on smartphone cameras every day — not to mention photo spheres, Google Street View, and other incredible innovations!
After condensing the code from Part A into dedicated computeH
and warpImage
functions, the provided starter code was used to obtain Harris corners for the center and right perspectives from above. These detected interest points (5830 and 6698 for the two perspectives, respectively) are overlaid below.
I then implemented ANMS to select 250 interest points from each image, shown below. These points, selected from the overall Harris output, have a noticeably more uniform spatial distribution across the images in comparison.
Here, I first extracted axis-aligned 8x8 patches for each window of size 40x40 in the image. Treating each of these multi-scale oriented patches (MOPS) as features, I then applied a threshold of 0.7 to match pairs with similar appearances, yielding a smaller set of points shown for each image below.
Finally, I applied RANSAC to select the final points for alignment, computing robust homography estimates for each image. After to applying geometric constraints and rejecting remaining outliers, RANSAC run with an epsilon value of 4.0 yielded the four points shown below for each image.
Blending the two perspectives with these interest points yields the raw mosaic below, obtained using container functions for the approach employed in Part A.
For reference, manual alignment yielded the following raw mosaic for the same input perspectives in Part A.
Cropping this output yields the following image, with the result containing minimal indications of panoramic blending. This output is a noticeable improvement over the manually aligned images of Part A, with the only blemish being a slight misalignment along one of the sidelines of the basketball court.
The algorithm also performed well when blending the center with the left perspective in place of the right, with the raw and cropped outputs shown below. Again, only a single slight blemish exists along one of the shadows present on the basketball court.
For the final of my three mosaics, I blended both of the outputs shown above to obtain a full panorama of the scene, combining the left, center, and right perspectives in the process. This also yielded strong results, with the lone blemishes already present in the inputs as a result of prior mosaicing operations.
What you've learned: I really enjoyed the practical nature of this part of the project, as it involved implementing findings from a publication. This is a skill which can be applied in countless forums between industry and academia, and it was great being able to practice it to bring image autostitching to reality.