This project involves calculating a homography matrix in order to warp images into a different shape.
Shoot the Pictures
For this part I started with some sample pictures I found of Shattuck at Berkeley. They are taken from the same position with each image taken from a different rotation on that position.
Left most imageRight image
Recover Homographies
There are 8 degrees of freedom for a homography, so the homography matrix is represented by a 3x3 matrix, with a constant of 1 at the very end. The matrix is supposed to be a solution to the equation p' = Hp. To calculate the entries of the homography matrix, H, a system of linear equatinos is set up as shown below.
A system of linear equations that will be solved with least squares
Warp the Images
The resulting H matrix can be used with remap in order to warp images with interpolation. The warped image may be larger than the original, so padding was added to the image to account for this. 6 points from both images were selected to calculate the homography. The left image was warped to the points from the right image, so the right image stayed the same.
Left image warped
Image Rectification
To test if my warping function was working properly, I took a few image with planar objects and "rectified" them by defining an arbitrary corresponding set of points to create a homography for. For the tiles example, I knew the tiles were square in shape, so I selected four points encpasulating a square section of the ground and calculated a homography to an arbitrary square. The painting example used the same logic, selecting the four corners of the painting and calculating a homography to an arbitrary rectangular shape of around the same proportions.
Original imageRectified tiles image based on the groundOriginal imageRectified painting image
Blend the Images Into a Mosaic
This next part was a bit trickier. Since the resulting image from the homography could fit outside the original image dimensions, padding was added to the two original images. This way if the result was shifted it would still be within frame. To calculate the homography, the indices of the new image were mutliplied with the inverse of the homography obtained earlier in order to obtain the corresponding coordinates of the source image to pass into cv2.remap that would do the interpolation.
Below is the left warped image blended with the right image, which is unchanged.
What I've Learned
This project really helped me understand the uses of homography as well as how to compute the homography matrix. Warping was a bit more difficult, but it was a rewarding seeing how the homography matrix played into retrieving the source coordinates.
