Arjun's CS294-26 Proj4a

Part 1 Rectified Images

I took images of tilted planar objects. Labeled the four corners and warped those out to the corners of a rectangle who's aspect ratio seemed to correspond roughly with the true objects. The results shown below seem pretty nice. One comment is with the student flex pass banner it was a little wrinkled on the right side. As a result when we rectify these wrinkles persist and are responsible for the interesting effect on that side of the image.




Part 2 Mosaics

For each of three scenes on campus I took a series of 4 photos rotating my phone camera around a vertical axis. For each pair of consecutive images I labeled 10 correspondences and used least squares to come up with the homography. (Math and example for this below.) I started from the outside of the sequence and moved inward each warp+blend leaving one image unchanged and warping the other into its perspective. Ultimately one of the middle frames decides the final perspective as a result. For blending I used an enhanced version of the laplacian pyramid strategy from Project 2. To find the initial binary mask, I considered the centroids of the two images and segmented points based on which centroid they were closer to. After using the laplacian stack algorithm I exclusively applied the gaussian masks to the areas where both images were available obviously only using the single image that covered other parts of the joint canvas. I blurred the initial boundary quite a bit prior to this process as I found that resulted in smoother transitions. Finally for all scenes I also present an automatically trimmed versions similar to what a smartphone may present when you take a panoramic image.

Zellerbach One interesting note about this mosaic is that there was a person walking below while I was capturing the images. This suprisingly worked out alright, we simply have (atleast) 2 distinct versions of the person visible!



VLSB One interesting note about this mosaic is I seem to have done a better job rotating purely around the vertical axis resulting in the untrimmed version closer to the trimmed one. With Zellerbach above there's is quite the fan out effect near the left and right sides.



Trees One interesting note about this mosaic is finding correspondences was more tricky due to the natural scenery. Nevertheless it worked out pretty well, meaning with enough correspondences a little noise isn't too bad.



Homography Math

Example of 10 correspondences.

For every pair (xa, ya) <-> (xb, yb) we have the equation:
[a, b, c]^T [xa ya 1] = w * xb
[d, e, f]^T [xa ya 1] = w * yb
[g, h, i]^T [xa ya 1] = w
Which implies the following two eqs once we eliminate w
a * xa + b * ya + c = (g * xa + h * ya + i) * xb
d * xa + e * ya + f = (g * xa + h * ya + i) * yb
Since we may simply set the scale factor i to 1 we get
a * xa + b * ya + c + g * (-xa * xb) + h * (-ya * xb) = xb
d * xa + e * ya + f + g * (-xa * xb) + h * (-ya * xb) = yb
That's two linear equations with 8 coefficients. Collecting 10 correspondences gives us 20 more than enough to apply least squares and get a reliable solution.

Overall one thing I thought I was pretty cool was how well we could blend together the different images to really make it feel like it came from a single capture. Granted some of the mosaics are better than others, but I felt I wasn't extremely careful taking photos making sure the camera didn't move so thought it was pretty cool that it came out as well as it did.