The goal of the project was to dive into the concepts we have been learning over the past few weeks such as homography and image warping. Utilizing homographies with image warping and blending techniques, I was able to create image mosaics at the end of this project.
Before beginning to code, the first task of the project was to find appropriate images that we could use. I need to ensure that I took the photographs in a way that allowed the transforms between them to be projective. I accomplished this by taking pictures from the same p.o.v. but with different angles. I used my Iphone with AE/AF lock to ensure that the setting would remain the same and not cause any issues later on.
One Example of the Images that I took (more can be seen below where I rectify and blend images into a mosaic):
To compute the homography matrix, I first collected points using Python's ginput function as we have done in previous projects. From there I was able to compute p'=Hp and find H. I did this by following an article that I found on Piazza: https://towardsdatascience.com/estimating-a-homography-matrix-522c70ec4b2c. This article discussed how we can recover our homography matrix for co-planar points. It essentially explained that two 2D images are related by a homography H, if both view the same plane from a different angle. So, I was able to follow the equation shown below.
Once, I had created the matrices I needed, I was able to utilize least squares to create my 3x3 homography matrix. Overall, this task helped me find the the homography matrix H that calculates the point(x',y') in the destination image which relates to the point(x,y) in the source image. It was amazing to see how we can realate two 2D images in the same plane even with different angles!
We can now use the homography we found in the last part and use it to warp our images. I was able to use inverse warping, I applied the inverse homography matrix to every pixel and applied interpolation as needed, this allowed me to warp my first image to my second. We can see the power of the homography function working with the warp below in the image rectification section!
This part of the project gave me an opportunity to check that my homography and warp functions were working correctly. All that I had to do was select my correspondence points as my image1 points and define another set of correspondences by hand as my image2 points, compute the homography matrix with these points, and lastly warp the image so that the plane was parallel. Examples of this can be seen below.
Here the goal was to attempt and try to create a frontal view of the further away posters, which still seemed to work well.
To create a mosaic, I utilized selected points along with the corners of each of the images to create a large enough canvas for the mosaic to render and display well. One thing to note, for my sunset mosaic it seems like the two cars in bottom right have clashed, but that only occurred because the cars were moving despite me taking them within seconds of each other.
During lecture I was not really able to connect how all these concepts would really work together. But it is amazing to see that with simply solving for the homography matrix and using warping we are able to change the perspective of the image quite easily. Additionally, it was awesome to get some insight to see how these panoramic pictures are created and made as I have used this feature many times on my Iphone and have always wondered how it was done