In this part, I chose two images per scene from a total of 3 scenes to join. I made sure that the two images in each scene had around a 50% overlap. In each scene, one image was taken from a straight on angle and the other was taken from a more tilted, bent right angle. The images taken were of the vending machine in my lift lobby, my roof, and my room. These images are shown below. They were all taken using my iPhone.
I also collected images for the future rectification part of the project. To do this, I collected slanting images of a textbook and a tissue paper box on a notebook. I chose these as the picture was taken at an angle and the depth of these objects was clearly perceivable so correctly rectifying them would be easy to spot if we were able to straighten, and flatten the surface of the books. These images are shown below. They were all taken using my iPhone.
In this Part, I knew that since we had 8 degrees of freedom I only needed 4 corresponding points from each image to define the transformation matrix. However, as we saw, overdetermined systems (> 4 points) work better when trying to recover the homography matrix. I used 8 corresponding points per image for all 3 scenes of mine when trying to recover the matrix. Since the system was overdetermined, I used least squares to solve for the matrix. The least squares construction was done in such a way that H*p is as close as possible to p' where p is the points in the first image and p' are the corresponding points in the second image. We set this up for 8 parameters in the 3*3 matrix and the bottom right entry was 1. Below are the corresponding points chosen for one pair of images. For the remaining scenes, you can reference the notebook to see the points chosen.
In this part, I created a warp function through using the polygon and matmul function. I did an inverse warp so I had to take the inverse of our H matrix. This was multiplied by a vector whose entries were determined by the polygon created using the transformed corners of our original image. This was found by H * original corners. The polygon gave me all the coordinates that I needed to inverse warp as it was contained within the transformed corners. I warped the 'side on' (second) image of all my three scenes. The results are shown below.
In this part, I used the previous parts of recovering a homography and image warping to rectify an image. More specifically, I found the homography between the 4 corners of an image and the 4 corners of my textbook. I then inverse warped the textbook using the homography that I found so that the resulting image was rectified and plane aligned. The results turned out well and can be seen below. Both images are squashed, straightened and plane aligned. The original images can be found in part 1.
In this part, the methodology I used was to blend the main straight on image with the 'side' image warped onto the main image. More specifically, to do this blending, I created a mask for both images where the pixel value was not 0 and then got a combined mask by taking the logical and for these two masks. Keep in mind, the size of the mask was the same size as the warped image. I then overalid both my images onto my result canvas making them overlap and used weighted averaging to effectively blend them together. The results for each of my three scenes are shown below. As we can see, the objects such as the box on top of the vending machine align well enough however there seems to be a blending issue with the colours. Perhaps this is because of slightly different lightings and camera settings during image capturing which can be taken care of with a more professional camera. Otherwise, the images turned out decently well.
I learnt a lot about how Homographies can be found and how extremely dependent and sensitive they are to defining good correspondences and not having any mismatch/outliers. The process was definitely time consuming and annoying which can hopefully be solved using automatic correspondence alignment. Furthermore, I once again learnt about the power of linear algebra in solving for transformations and in warping.