CS194-26 Intro to Computer Vision and Computational Photography Project 5a

Computing the Homography Matrix

The aim of this part is to be able to find the homography matrix between two sets of images. The way this is done is by first defining corresponding points between two images and then defining a system of equations that we can solve to retrieve the elements of the homography matrix. Recall that a homography matrix has 8 degrees of freedom since we assume the bottom right corner to be 1 since that is the scaling factor.
So the million dollar question is, how exactly do we set up the homography matrix equations? Well, we need at least 4 points to account for the 8 DOF's but it is often better to setup an overdetermined system and use least squares.

Here is the way I setup my system of equations and I have found using least squares gets me pretty good results, for simplicity I have only shown the first two rows (assume x,y are coordinates in the original image and x_prime, y_prime are coordinates we want to map to):

[x, y, 1, 0, 0, 0, -x_prime*x, -x_prime*y] = x_prime
[0, 0, 0, x, y, 1, -y_prime*x, -y_prime*y] = y_prime

Using this here is a sample Homography computed for the rectification part using my method (on top) and using cv2.findHomography (on bottom), as you can see there is lttle difference.

Warping Image

The warping procedue is very similar to that of the average face project, with one major difference: in this case we can actually warp to coordinates that are negative or greater than the image size. To counteract this we first estimate the size of the bounding box by mapping the four corners of the image and then padding the image with an appropiate offset. Additionally, I also compose the homography matrix with some offset transformation making the new homography matrix T@H, obviously nromalizing so scaling factor is still 1. With this in mind we can do a simple remap and write the whole function without for loops.

Rectification

For this part I decided to make my ipad a square. To do this I selected the four corners of the ipad and then manually inputted for points to make a square centered at (600,600) with 400 pixel sides. I have been struggling with my rectify function however this is what it currently looks like but hopefully I will be able to fix these issues in the next couple days.

Image Mosaicing

For this part I decided to select 8 points in both images so as to obtain a more accurate homography matrix. I then decided to project both of my images onto the plane defined by the average points and then stitch them together. For blending I have decided to implement a laplacian pyramid although I still need to fix that as well :( here are my current results: