In this project, I defined correspondances between images by hand and projected the images onto the same plane so that they could be blended together
I first found the following images that I decided to use for this project:
After specifying correspondances by hand, I found the projection matrix using least squares. I first defined vector $h$ to contain each element of the projection matrix in row major order and then defined matrix $A$ to contain the partial products between the correspondances of one image. Then, the matrix vector product $Ah$ is defined to be equal to an estimate of $x$, the correspondances of the second image.
This then allows us to calculate $h$ as $(A^TA)^{-1}Ax, using more than four correspondances.
In this section, I was able to warp images to a specific plane with the following steps:
1. Find the projection matrix og_H from images x to y using the steps in thre previous section
2. Use the projection matrix to calculate the size of the new image x'.
3. Take the inverse of og_H and multiply it with a vector of the coordinates of our new, empty image x'.
4. Interpolate the colors of x' from x using the results of og_H^{-1}x'.
Although I have a working implementation of these steps, relying on the remap function of the CV2 library ultimately worked best. Using these steps, I then retificated the following images below.
I learned a lot about how information from images can be manipulated. The coolest part of this project was rectifying the images, since the images looked somewhat natural in many cases—it seemed like just a difference in perspective. That was the coolest part of this project: understanding how much information is in an image and being able to take advantage of that information.