For part A of this two-part project, I explored homographies and tranforming images. I experimented with "rectifying" images by making them appear planar, as well as producing visually pleasing mosaics of multiple images. One of the mosaic pairs (of downtown Berkeley) was taken off of the Berkeley campus.
I detail the theoretical musings, approaches taken, and results for various experiments in these domains below.
I shot most of the pictures with a Nikon D3300, and I shot one of them with the iPhone 6S. For all the mosaic pictures, I took the pictures in pairs, and stayed at the same place (same center of projection), but shot at different angles.
I used Matlab's cpselect tool to manually generate correspondences between images. I played with the cpcorr tool, which uses local neighborhood matching to improve correspondences, but I found that not using the tool and just being very careful worked best when generating good correspondences. As a modification to this scheme; for image rectification, I used the vertices of a simple unit square as one of the correspondence point sets, which I scaled and shifted later, as appropriate.
I used skimage.io's warp which was stated to be permitted on Piazza. I experimented with different kinds of interpolation, but found that default bilinear interpolation was fast and usually very good.
In this section, I "rectified" a chosen surface which I knew was flat intuitively, projecting it onto a planar surface such that it the rectified image appeared to be viewing the surface from above. I marked out correspondences between the corners of the chosen surface, and those of a unit square, which I later scaled and shifted (tuning by eye). I then transformed the surface to the overhead planar view by finding the homography matrix and appropriately warping.
Original roof image:
Rectified roof image:
Original scones image:
Rectified scones image:
The rectified images look very nice! I noticed there is a small amount of aliasing in the images, which is expected as we are interpolating more information than we have by doing these warps. The biggest failure case occurs (as is visible in the roof rectification) when there is some depth perspective in the original surface, since it will not disappear after the warp as it should.
In this section, I projected one image taken at the same center of projection (but different viewing angle) onto the plane of another, forming a mosaic. I manually inputted correspondences between the images and calculated the homography matrix. I then warped one image into the other, and then generated a mosaic by taking a weighted sum of the two final images. The result had some lighting inconsistencies, so I applied bilinear gradient masks to each section of the mosaic ( uniquely image 1, uniquely image 2, and the overlap region) and tuned the mask parameters until I obtained a visually pleasing mosaic.
Original campus images:
Campus mosaic:
Original downtown images:
Downtown mosaic:
Original rooftop images:
Rooftop mosaic:
The mosaics seem very nice! The only mosaic that looks problematic is the last one; notice how the overlap region for the two original pictures is both small and sort of blurry (because of the blowing leaves, I presume). This resulted in a good stitching near the overlap locality, but far away there were some clear issues (such as the slanted wall). The only other issues seem to be small lighting-difference edges and some small aliasing effects. Ideally, I could have done multiresolution blending to smooth out these artifacts, but they are already very small so I decided it was not worth the time. I found in this section that very accurate correspondences (and many of them!) are required. Correspondences being off just by a little bit could completely throw off the homography.
I learned a lot about actually implementing homography mosaic-ing! The most important and surprising thing was that correspondences that are just a little bit off can completely screw up the resulting homography. Since the most tedious part of the project is selecting these precise correspondences, perhaps the most important avenue to look into moving forward is ameliorating this process.