Programming Project #4 (proj4A) (first part) (Ujjaini Mukhopadhyay)

Recover Homographies

To recover the homography, I solve a system of equations. I know that applying a homography warps an image so I can do somehting where p' = Hp where H is a 3x3 matrix. I know that the bottom right corner must be 1, so I have 8 degrees of freedom. I can solve this using 4 points since each point gives me two pieces of information, but it is better to solve this as a least squares problem. I first define correspondences on my images, and form a matrix that looks something like this:

Warp the Images

To warp the images, I take in my image that I want to warp and my calculated Homography. I want to do inverse warping so that there are no holes in my final object. Thus, I calculate the inverse of the homography, find all the points on my final image that I want to warp and remap them accordingly. I took a lot of inspiration from https://stackoverflow.com/questions/46520123/how-do-i-use-opencvs-remap-function and piazza.

Image Rectification

To show that my warping works, I rectified the following image to make it seem frontal parallel.
Here is the original image:

Here is the warped image after selecting four points and setting their final points to be square:

As you can see, this is the frontal parallel version. I also did the same with another image:

Here is the warped image after selecting four points and setting their final points to be rectangular (kind of in proportions of the iPad) -- also, Go Adele!:

Blend the images into a mosaic

Now that I can warp, I can build larger canvases, warp to them, and display panoramas. The method I chose is to constantly warp the left images and add in the right image as I go through a list. I was always calculating the corners and if any of the left corners were negative, I added an x_offset and if any of the y values were negative, a corresponding y_offset as well. For blending my seams, I used Gaussian Blending. To create the mask, I used polygon between my corners (brought the corners in a little bit so that the edges are less obvious) and created a gaussian stack.
Here are some examples:

Adding on another photo, this is my final panorama:

Through this project, I learned the importance of defining correspondences correctly. I put together 3-4 more panoramas which all failed becuase the pixels weren't exact. Had I had more time, maybe I would have gone back to fix the correspondences. I tried variations of choosing 4 points, 10 points, even up to 30 points and used Least Squares solutions to solve, but even so, small mistakes or even un-compatible pictures made the panoramas very difficult to put together.